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Final Report 

Non-Deterministic, Non-Traditional Methods (NDNTM) 


Executive Summary 
Review Context 

The purpose of the study was to identify research opportunities related to the use of non- 
deterministic, non-traditional methods to support aerospace design. The scope of the study was 
restricted to structural design rather than other areas such as control system design. Thus, the 
observations and conclusions are limited by that scope. 

The review identified a number of key results. The results include the potential for NASA/AF 
collaboration in the area of a design environment for advanced space access vehicles. The 
following key points set the context and delineate the key results. 

The Principal Investigator’s (Pi’s) context for this study derived from participation as a Panel 
Member in the Air Force Scientific Advisory Board (AF/SAB) Summer Study Panel on “Whither 
Hypersonics?” A key message from the Summer Study effort was a perceived need for a national 
program for a space access vehicle whose operating characteristics of cost, availability, 
deployability, and reliability most closely match the NASA 3"* Generation Reusable Launch 
Vehicle (RLV). The Panel urged the AF to make a significant joint commitment to such a 
program just as soon as the AF defined specific requirements for space access consistent with the 
AF Aerospace Vision 2020. 

Kev Findings: 

The point this study brought home is the concurrent need for a national vehicle design 
environment. Engineering design system technology is at a time point from which a revolution as 
significant as that brought about by the finite element method is possible - this one focusing on 
information integration on a scale that far surpasses current design environments. The study 
therefore fully supported the concept - if not some of the details - of the Intelligent Synthesis 
Environment (ISE). 

It became abundantly clear during this study that the government (AF, NASA) and industry and 
not moving in the same direction in this regard - in fact each is moving in its own direction. 

• NASA/ISE is not yet in an effective leadership position in this regard. However, NASA 
does have complementary software interoperability efforts that should be a part of any 
major ISE program. Software standards that assure interoperability of data systems and 
modeling representations are enabling for the proposed research advocated herein and 
should be a major element in the ISE initiative. 

• The international standard for data interchange is known by the acronym “STEP.” 
The NASA participation and lead for that effort is at the Goddard Space Flight 
Center. 

• NASA/GRC is leading an effort to define CAD geometry standards through the 
Object Management Group (OMG). 

• To enable the design environment so necessary to the above national vision for a unique 
space vehicle will require an integrating software environment with interoperability 
standards that allows the development and widespread deployment of tools and toolsets, 
rather than traditional “shrink-wrapped” software used by engineers today. 
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• Someone or some agency has to define that software environment and its standards in 
order for the US aerospace industry to approach decent levels of interoperability of 
design tools and design information. NASA has some key efforts in this area. 

• Each aerospace company today appears to be developing its own version of an ISE. 

These are not inter-operable environments. 

• The AFRL has its own program called the Common Engineering Environment (CEE). 

The AF, NASA and industry could cooperate to develop a true, common design environment in a 
way that will benefit the nation later on. NASA is the obvious agency to pull this together. 

The contracted task of Non-deterministic, Non-traditional Methods (NDNTM) is but a subset task 
within a design environment. 

• A key NDNTM finding is that there is no current need for new probabilistic methods but 
rather for tools and toolsets to be developed that would support the deployment and use 
of non-deterministic methods. 

Another key finding is that the required accuracy for reliability assessment to support the fleet 
risk assessments that will underpin the future design environment poses several key issues that 
must be addressed within NDNTM development. 

• We do not have the necessary tools and toolsets to deploy today’s probabilistic 
technologies for real systems with flight safety requirements. 

• We do not have the needed error bound technology base for the various non- 
deterministic methods. 

• We do not have the design-enabling environment to support the full deployment of 
non-deterministic design outside of the research office. 

• There is consensus among the industry experts consulted by the PI on these points. 

The NDNTM study therefore focused on the issues for enabling this technology base for large- 
scale aerospace system design. The study concludes by recommending specific programs to 
support the deployment of non-deterministic design in ways that will be accepted and used by a 
still-reluctant aerospace design community. The following are the priority-ranked 
recommendations for NDNTM development. 

1 . Probabilistic Data Expert (PDE): The PDE provides a systematic means for merging 
disparate forms of “data” into probabilistic design. The data ranges from subjective 
expert opinions to raw data such as specimen specific fatigue test data. 

2. Probabilistic Updater (PU): The probabilistic design process for total life cycle 
design from concept to field deployment for complex systems generally involves 
reliability growth processes of one form or another. Semi-automated systems to 
support model updating based on information and data that become available after 
the initial design are required. 

3. Probabilistic Error Bounder (PEB): A key and, as yet, not addressed and 
fundamentally critical need is to be able to compute rational reliability error 
prediction bounds for the various algorithms likely to be used an advanced aerospace 
vehicle and propulsion design. Error bounds are needed that span the analysis 
spectrum from the physics approximations, to the response surface or surrogate 
models used to represent the physics, to the specific probabilistic algorithms 
deployed. 
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4 Probabilistic Model Helper: A major limitation in the deployment of probabilistic 
methods for system design is the current burden that the analyst be a skilled 
“probabilistic engineer.” Non-traditional methods provide a real basis for the 
development and deployment of intelligent systems to work as probabilistic “robots” 
or assistants. 

5. Probabilistic Mesomechanics: Significant progress has been demonstrated in the 
ability to link material processing simulation software with micromechanical models 
of material behavior at the mesoscale (grain size, grain orientation, flow stress, 
dislocation density) to predict statistical distributions of material fatigue strength. 
Such probabilistic modeling advances lend real credibility to the notion of engineered 
materials that goes well beyond the simpler notions associated with composite 
material systems. Probabilistic mesomechanics tools that can support the PDE tool or 
feed “data” directly into the material data distributions are required. 

6. Response Surface Generator (RSG): T wo methods are currently employed to 
generate RS’s; these are the Taylor series method and the “design of experiments 
(DOE)” method. RS methods are highly effective over a range of design levels from 
conceptual to final. A generalized “tool” for effective development and 
representation of RS’s is needed that will interface both to probabilities and to the 
closely related field of multidisciplinary optimization (MDO). 

7. System Reliability Interface: The aerospace system design environment of the future 
requires the ability to interface multiple models of multiple subsystems in an efficient 
and accurate manner. Intelligent systems can be developed for linking the 
information and propagating it to the various “top level” events in order to provide a 
powerful environment for calculating and managing system level reliability. 

8. Health Management System: A new Health Management (HM) design strategy 
should allow inclusion of sensors and diagnostic/prognostic technologies to be 
incorporated early in the design process to optimize system performance and life 
cycle costs. 
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1.0 Detailed Overview of the Effort and Report 

The purpose of the study was to identify research opportunities related to the use of non- 
deterministic, non-traditional methods to support aerospace design. The scope of the study 
was restricted to structural design rather than other areas such as control system design. Thus, 
the conclusions are limited by that scope. 

Several positive developments characterize the developing aerospace systems design 
environment based on the observations made during this review. The design environment is 
undergoing revolutionary changes that are at least as significant as the change from hand 
analysis to ubiquitous application of finite element methods that began roughly 40 years ago. 
These changes are associated with web-enabled, distributed computing and with object- 
oriented programming. Every major aerospace airframe and engine company is actively 
working on large-scale, integrated design systems with extensive generalized database (what 
will be referred to as information) linking capabilities. 

The fundamental common element in these generalized approaches is information linking 
where disparate types and forms of information are made easily accessible to the various 
specialized tools used for design analysis. The enabler for the developing design environment 
is object-oriented programming, specialized tools for data creation, visualization, and 
analysis. So-called data wrappers are specialized tools that automatically create links between 
analysis tools and the required analysis information. 

The new software environment is increasingly web-enabled. Web enabling includes the use 
of Java-based graphical interfaces such that the workgroup on design projects may share 
information and resources over the Internet. Web-enabled tools mean the programming tasks 
can be done once and used over a wide range of processors. The tool sets that are being 
deployed often allow the individual user to tailor their own interface to support specific 
analysis or interpretation purposes. 

These developments are taking place generally outside the context of NASA’s Intelligent 
Synthesis Environment (ISE). The ISE initiative appears to be a result of these developments 
rather than a leader of these developments. Rather, industry has taken the lead in moving to a 
new design environment under pressures of product development costs and cycle-times, 
recognizing that improved information processing is the underlying enabler of true 
concurrency in design. However, the very fact that each company is developing their own 
environment guarantees that information sharing required to complete a new vehicle system 
design for NASA will encounter significant information barriers at the system level. 

Finding: NASA has an opportunity under the banner of ISE to provide leadership to the 
aerospace community by taking leadership in creating high-level information and 
programming standards that can facilitate advanced space vehicle design requirements. Such 
an integrated system is seen as a fundamental requirement for the successful execution of the 
proposed 3 rd Generation Reusable Launch Vehicle, for example. Defining such standards 
would enable tool development and information processing that can be shared between 
multiple proprietary systems. To date, there is no evidence that the NASA ISE initiative has 
taken this leadership role. 

Finding: NASA has demonstrated leadership in the effective deployment of system 
integration platforms that enable the aerospace propulsion community to integrate proprietary 
and NASA-developed codes through the NPSS effort. NASA continues in some areas, 
however, to pursue multipurpose, stand-alone codes that do not integrate well with other tools 
or codes. 
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Recommendation: If NASA were to take the recommended approach to ISE by facilitating 
an aerospace industry integrating design environment, NASA would also have the ability to 
promote the development of tools that can easily be integrated by the various aerospace firms, 
contractors, and universities into this design environment. This report recommends the 
development of several key, object-oriented software tools or toolsets that build on non- 
traditional design methods and facilitate the widespread application of non-deterministic 
design methods that will be critical to the success of a new space vehicle design program. 

Such tools will achieve NASA’s goal of deploying NASA-logo bearing products to the 
design workplace if they employ modern, object oriented programming methodology. 

A critical design requirement for future space vehicle designs is high system reliability over 
the full life cycle of the system. Non-deterministic design methods are an intrinsic element in 
meeting this requirement. A major element in the completed review of NDNTM has been to 
assess the current state of probabilistic methods for structural design. The information used to 
make the following finding is based on the following sources: 

• The ongoing efforts by the Society of Automotive Engineers (SAE) Committee G-l 1 
Probabilistic Methods (PM) Committee. That committee effort includes assessment 
of the technical quality and capabilities of various probabilistic methods. The work of 
this group can be referenced at 

http://forurns.sae.org/access/dispatch.cgi/TEAG1 1PM pf . 

• Reviews conducted by the USAF as part of the current contract on probabilistic high 
cycle fatigue (HCF) under the leadership of Stress Technologies, Inc. 

• Input from various probabilistics experts including Southwest Research Institute, 
Unipass Software Systems, ARA Inc., and members of the technical staffs at Boeing, 
Pratt & Whitney, Rockwell, Honeywell, and GEAE. 

• Proceedings of the AIAA Nondeterministic Methods Forum conducted in Atlanta GA 
on 3 April 2000. 

• Recorded comments from the AIAA Nondeterministic Methods Panel in Atlanta GA 
on 4 April 2000. 

Finding: The various algorithms for probabilistic structural analysis are diverse in structure, 
approach, and capabilities. The algorithms themselves are still the subject of intense scrutiny 
and debate. However, there is consensus that there are no substantive research opportunities 
at this time for new or improved probabilistic algorithms, per se. 

Finding: The integration of probabilistic structural analysis algorithms into system design is 
primitive at best. Significant research needs and opportunities have been identified that will 
move probabilistics from the research application to vehicle system design capability. The 
needs will draw on technologies developed in the general area of NDNTM. 

Figure 1 illustrates a nondeterministic design environment based on today’s probabilistic 
design codes, shown at the center. The current study has found widespread interest in and 
need for powerful “tools” that will provide critical interface capabilities between the 
designers and the current probabilistic algorithms. The various tools (meant in the broadest 
sense) include the following items. 

Probabilistic Data Expert (PDE): The PDE provides a systematic means for merging 
disparate forms of “data” into probabilistic design. The data ranges from subjective expert 
opinions to raw data such as specimen specific fatigue test data. The PDE relies on non- 
traditional methods for integrating “fuzzy data” based on qualitative information. The major 
technology needs here are in creating tools for automating the data acquisition and integration 
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processes. A pilot study for probabilistic HCF has been forwarded to NASA under the ISE 
banner as part of the joint AF/Navy/NASA and Industry program. However, there are 
additional basic research needs in addition to the proposed pilot study. 

Probabilistic Updater (PU): The probabilistic design process for total life cycle design from 
concept to field deployment for complex systems generally involves reliability growth 
processes of one form or another. Early in the design process, the reliability characteristics of 
the subsystems are either not known or crudely known. Testing during development should 
focus on those results that are most effective either in demonstrating greater reliability or 
increasing the assurance (reducing the uncertainty) in the reliability prediction. There exists a 
critical need to provide a generalized tool that will provide intelligent ways to update the 
current reliability and assurance intervals (akin to statistical confidence intervals) taking into 
account new information and data. Reference [33] discusses these intervals. 



Figure 1: Integrated Nondeterministic Design Environment 

Probabilistic Error Bounder (PEB): Many systems design processes have allowed for 
generalized levels of comfort in product performance - so-called “fuzzy” non- 
deterministic output. However, both the commercial workplace and the demands of 
government acquisitions have transformed the aerospace vehicle design requirements 
from non-analytical reliability bases to specified levels of “demonstrated” reliability that 
may include both test and analytical bases. A key and, as yet, not addressed and 
fundamentally critical need is to be able to compute rational reliability error prediction 
bounds for the various algorithms likely to be used an advanced aerospace vehicle and 
propulsion design. Error bounds are needed that span the analysis spectrum from the 
physics approximations, to the response surface or surrogate models used to represent the 
physics, to the specific probabilistic algorithms deployed. In fact, it is likely that 
engineers will need to deploy hybrid combinations of the current probabilistic methods 
(e g., FPI, Monte Carlo importance sampling) for large, system design problems. Error 
identification, tracking, accountability, and bounds are required for the full process. 
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Response Surface Generator (RSG): All current probabilistic algorithms for any but the 
most trivial problems use a “response surface” or surrogate model to describe the 
component, sub-system, or system level behavior as a function of the design variables. 
Thus, it is a surface in the N-dimensional design space of the problem. In fact, it is 
increasingly likely to be a union of many such hyper-surfaces that can be used for 
complex system design. An example of a powerful algebraic system for manipulation of 
RS’s that could be easily extended to non-deterministic design is seen at the Rockwell 
Science Center link: http: //ww w. rsc. roc k wel 1 . com/des i gnsheet/ . 

Two methods are currently employed to generate RS’s; these are the Taylor series 
method and the “design of experiments (DOE)” method. The former are more likely to be 
employed near a critical design point while the latter are more often used to span a wider 
range of the design space. Fast probability methods such as the Fast Probability Integrator 
(FPI) algorithms or importance sampling (a Monte Carlo simulation strategy) are 
performed using the RS and do not directly use the actual detailed simulation. As such, 
the RS methods are highly effective over a range of design levels from conceptual to 
final. A generalized “tool” for effective development and representation of RS’s is 
needed that will interface both to probabilistics and to the closely related field of 
multidisciplinary optimization (MDO). Further, the PEB tool and the RSG tool should be 
able to work together to adaptively build optimal RS strategies that combine 
computational efficiency with minimizing the probabilistic approximation errors for large 
system design problems. 

Probabilistic Model Helper: A major limitation in the deployment of probabilistic 
methods for system design is the current burden that the analyst be a skilled “probabilistic 
engineer.” Such individuals at this time are probably numbered in the low dozens and are 
scattered between software firms, universities, government labs, and industry. The field is 
not one that has received much attention in academic curricula although the topic is now 
covered in the most recent undergraduate mechanical design texts. Further, the process of 
preparing and interpreting a probabilistic design problem involves the inclusion and 
assessment of many more sources of information and decisions regarding the analysis 
strategy. Non-traditional methods provide a real basis for the development and 
deployment of intelligent systems to work as probabilistic “robots” or assistants. The 
approach will likely require significant use of adaptive networks, software robots, genetic 
algorithms, expert systems and other non-traditional methods. 

Probabilistic Mesomechanics: Significant progress has been achieved since the early 
attempts at what was called “level 3” probabilistic material modeling as part of the 
original NASA Probabilistic Structural Analysis Methods (PS AM) contract. 
Demonstration problems have shown the ability to link material processing simulation 
software with micromechanical models of material behavior at the mesoscale (grain size, 
grain orientation, flow stress, dislocation density) to predict statistical distributions of 
material fatigue strength. Such probabilistic modeling advances lend real credibility to 
the notion of engineered materials that goes well beyond the simpler notions associated 
with composite material systems. Probabilistic mesomechanics tools that can support the 
PDE tool or feed “data” directly into the material data distributions are required. 
Automated, intelligent systems are envisioned with the capability to forward model 
material processing to define scatter in properties as well as sensitivity links to the 
independent process and material primitive variables. Further, the inverse problem of 
optimizing the processing and design of the material microstructure provides the ultimate 
in an “engineered materials” design capability. 
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System Reliability Interface: The aerospace system design environment of the future 
requires the ability to interface multiple models of multiple subsystems in an efficient and 
accurate manner. Each failure mode of the system has its own probabilistic model in 
terms of a response surface or surrogate model representation of the physical problem 
together with the associated non-deterministic design variable descriptions. Current 
NASA technology focuses on traditional block-diagram forms for representing system 
reliability. Each block is typically represented by point estimates of reliability that are not 
linked to the underlying physics or to the distribution and confidence in the underlying 
variables. Physics based system modeling provides an information-based opportunity for 
linking the key design parameters and physics-based models together from the sub- 
system/component level to the full system level. Intelligent systems can be developed for 
linking the information and propagating it to the various “top level” events in order to 
provide a powerful environment for calculating and managing system level reliability. 

Health Management System: Health Management is a philosophy that merges component 
and system level Health Monitoring concepts, consisting of anomaly detection, diagnostic 
and prognostic technologies, with consideration to the design and maintenance arenas. 
Traditionally, Health Monitoring design has not been an integral aspect of the design 
process. This may be partly due to the fact a cost/benefit model of a HM system 
configuration cannot be fully realized at this stage. Without a doubt. Health Monitoring 
technology must “buy” its way into an application. Hence, the need exists to extend the 
utility of traditional system reliability methods to create a virtual environment in which 
Health Monitoring architectures and design tradeoffs can be evaluated and optimized 
from a cost/benefit standpoint. This capability should be present both during the design 
stage and throughout the life of the system. A new HM design strategy should allow 
inclusion of sensors and diagnostic/prognostic technologies to be generated in order to 
produce an enhanced realization of component design reliability requirements at a very 
early stage. Life Cycle Costs can be reduced through implementation of health 
monitoring technologies, optimal maintenance practices and continuous design 
improvement. To date, these areas have not been successfully linked with non- 
deterministic design methods to achieve cost/benefit optimization at the early design 
stage. 

Summary Recommendation: Significant opportunities and critical needs exist in the 
area of non-traditional methods as applied to the area of non-deterministic modeling to 
support reliability-based design for advanced, complex aerospace systems. While the ISE 
initiative has the overall responsibility for the environment, the current NTNDM contract 
effort clearly establishes an opportunity for NASA/GRC leadership in the development 
and deployment of interoperable modeling tools that will substantially enable the 
deployment of non-deterministic capability to the aerospace industry. NASA/GRC is 
urged to define a long-range plan (5 to 10 years at the current level of funding) to fund 
the above tool needs. 
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2.0 Proposed NASA Research Announcement (NRA) Focus Areas 

2.1 Probabilistic Data Expert 

2.1.1 Background 

Probabilistic system analysis capabilities for complex aerospace systems must be 
effective elements within generalized design environments for integrated vehicle 
and propulsion systems. The information required to support such integrated 
design systems will characterize the uncertainties inherent to the type of design 
data (e g., loads, materials, etc.) at a level of fidelity consistent with the 
knowledge available at the particular stage of the design process (e g., 
conceptual, preliminary, final, in-service) [1]. Some of the design information 
will consist of usual forms of materials data such as “A” or “B” allowables and 
the supporting statistical data base such that probability distributions and 
confidence levels can be assigned to the data. Some of the design data will 
involve engineering judgment based on experience [2], Such data is “fuzzy” data 
in that there may be significant reliance on subjective measures and estimates. 
Some of the design data may be in the form of qualification testing at the sub- 
system or component level. 

The inherent range in design data from very subjective to fully characterized 
statistical models must be included in the probabilistic design analysis cycle. 
However, subjective data must be mapped into probability distributions to 
support the design process in ways that most correctly capture the expertise of 
the individuals submitting the data while removing to the greatest extent possible 
biases and unrelated judgments of the experts. Such “fuzzy” input data must then 
be effectively merged with the other types and quality of probabilistic input data 
so that the usual formalisms of probabilistic design analysis can be deployed. 

2.1.2 Statement of Work 

A comprehensive “tool set” is to be developed that accepts a full range of input 
data from domain-expert opinions to crisp statistical data and creates 
probabilistic data sets that define distributions (closed-form to empirical), 
uncertainty characteristics for the distributions, truncation levels, etc. such that 
comprehensive probabilistic design analysis can be performed on the created 
input data. Further, the tool set will provide linkage such that the resulting 
probabilistic design analysis distribution and levels of assurance (equivalent to 
statistical confidence levels) can be related back to the specific information 
elements for the un-mapped, raw input data. When data is taken from physical 
test results or from analysis models, the data is to include estimates of 
experimental error, modeling error, and any biases in these errors. Finally, the 
provided probabilistic design analysis input distribution data (known as “prior” 
distributions) are to be accessible through the tool set to a range of updating 
strategies. The technical elements to be included in the tool set are as follows. 

• Tools and strategies that support self-elicitation of expert opinions for the 
knowledge base for pilot applications. 

• Graphical tools specifically tailored to define design system processes 
and interactions, logic models, event trees, reliability block diagrams, 
Bayesian networks, FMEA models, etc. with the facility to generate 
appropriate data structures for subsequent application. 
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• Knowledge base groupware with extensive capability for segregating 
proprietary databases, tracking knowledge use to and from the 
application software to be used on pilot problem. 

• Mapping tools for converting knowledge base into prior distributions 
with traceability. 

2.1.3 Software Requirements 

All software is to be prepared using “C++” or “JAVA” based on the most up-to- 
date object oriented programming methods. The tool set is defined to mean the 
collection of all software objects required to provide tailored user interfaces, 
analysis modules, data interfaces, module linkages, and output tailoring including 
comprehensive graphics. The tool set is to be provided with non-proprietary 
“data wrappers” such that the input and output results are able to interact with 
text, spreadsheet, free-format, and other forms of data records as well as be able 
to prepare data for commercial analysis codes in the disciplines of structural, heat 
transfer, CFD, and algebraic interpreters. The underlying analysis software is to 
be capable of execution on NT workstations and Unix workstations. The 
interfaces are to be written in Java to be web-enabled and device independent. 

2.2 Probabilistic Updater 

2.2.1 Background 

Probabilistic design analysis for complex aerospace systems is required to 
support the entire life cycle of the system from conceptual design to service use. 
Knowledge of system variables and operating characteristics will increase in 
quality through the life cycle. Development testing is an inherent part of the life 
cycle of these complex systems and is intended to provide cost-effective means 
for improving the state of this knowledge. Improved knowledge and the 
associated development costs must be fully integrated into the probabilistic 
design analysis process for these complex aerospace systems. The integration 
must be a proper blend of automation and judgment such that prior distributions 
are updated for continuing system assessments of performance reliability 
consistent with sound engineering principles. 

Bayes theorem [3] is one but not the only method for updating. When the 
situation is appropriate for its use, it is often method of choice because of a most 
desirable result: when the prior distribution and the likelihood reinforce one 
another (overlap), the posterior distribution from the theorem has a narrower 
variance. One can overcome some of the major criticisms of the use of Bayes’ 
Theorem as with today's computing we are not limited to using specific 
distribution forms (e.g., beta) for computational convenience. 

By the same reasoning, we are not limited to using Bayes’ Theorem when other 
weighting schemes or conditional probability combining schemes are 
appropriate. For example, when two sources of information, say two reliability 
distributions from two experts, greatly differ, one would not blindly just use 
Bayes’ Theorem to combine these. First, as part of the formal elicitation methods 
(see Probabilistic Data Expert tool above), one would want to try and resolve 
why the two experts disagreed. Perhaps they had different assumptions or were 
assuming certain conditions were holding. If such differences cannot be resolved, 
then the large uncertainty range covered by both distributions reflects the state of 
knowledge at that time. It then becomes the task of trying to identify how to 
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gather additional information to reduce that large uncertainty. In the mean time, a 
simple weighting scheme (the default being equal weights) would be a method 
for combining the two different distributions. 

Regardless of the combining method used (Bayes’ or other), the recommended 
approach is to review all combining analysis results with the experts, to make 
sure that the combination is consistent with the existing state of knowledge. The 
elements of this process can be implemented in “smart” tools. 

2.2.2 Statement of Work 

A comprehensive “smart tool set” is to be developed to provide probabilistic 
updating capabilities to probabilistic design analysis systems. The tool set is to 
include a knowledge base as well as an adaptive network to support the 
“intelligent” use of various updating strategies and to advise the user of options 
or recommendations. The tool set is to operate on predefined networks of the 
system problem (see 1.1) such that measured results at any node can be used to 
support an updating of related priors. The tool set is to be configured in such a 
manner as to permit its application to single node models as well as system 
models with up to 10 5 degrees of freedom. 

2.2.3 Software Requirements 

All software is to be prepared using “C++” or “JAVA” based on the most up-to- 
date object oriented programming methods. The tool set is defined to mean the 
collection of all software objects required to provide tailored user interfaces, 
analysis modules, data interfaces, module linkages, and output tailoring including 
comprehensive graphics. The tool set is to be provided with non-proprietary 
“data wrappers” such that the input and output results are able to interact with 
text, spreadsheet, free-format, and other forms of data records as well as be able 
to prepare data for commercial analysis codes in the disciplines of structural, heat 
transfer, CFD, and algebraic interpreters. The underlying analysis software is to 
be capable of execution on NT workstations and Unix workstations. The 
interfaces are to be written in Java to be web-enabled and device independent. 

2.3 Probabilistic Error Bounder 

2.3.1 Background 

The use of probabilistic methods for gas turbine engine design will require 
efficient and accurate algorithms for low probability of occurrence design 
conditions. The engineer must have access to automatic algorithms and protocols 
that provide user-control over accuracy within the constraints of available 
modeling capacity and time. In order to do this, the various computational 
algorithms for reliability assessment need to have formal error estimators. 

Further, given the likelihood that hybrid methods are going to be required, hybrid 
reliability computation strategies must be developed and qualified. 

Fast probability integration (FPI) algorithms include one, two, and three 
parameter fits to the equivalent normal distributions. Some error studies have 
been performed. The errors are dependent on the various characteristics of the 
actual distributions such as uniform and truncated distributions. Some algorithms 
such as the fast Fourier transform (FFT) are better suited to certain types of 
distributions. The need is for a systematic study of accuracy and efficiency for 
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various, useful distributions. An analytical estimate of residual errors based on 
available information regarding the distributions being modeled is required. 

The iterative advanced mean value (AMV+) algorithm is taken to be the full 
iteration process at the estimated and updated design point. The requirements for 
this methodology are a mathematical demonstration of convergence to the “true” 
answer and a residual error term for both the first-order reliability method 
(FORM) and the second-order reliability method (SORM). Since these methods 
are implicit, the use of nonlinear mapping functions is not to be considered. 
Computational efficiency vs. accuracy is to be demonstrated. 

The response surface method has three areas of error estimations. The first error 
issue is the error estimates for fitting the response surface. The issues include 
selection of the fitting points and this should include DOE considerations as well 
as COV data for each variable. The second concern is the error estimation for the 
Monte Carlo method itself. Finally, the use of response surfaces for probability 
levels that are low (or high) focus on accuracy of the response surface near the 
portion of the design space that is driving that reliability calculation. Thus, local 
fitting issues rather than global have to be included in the error study and 
resulting metrics. Accuracy and computational efficiency studies are to be 
performed. 

The “best” solution is likely to involve combinations of algorithms in ways that 
focus the process of getting to the region of the final design point quickly and 
resolving the estimated probability for the design point with user-defined 
accuracy. Residual error estimates are needed. Computational efficiency with 
specified error levels is to be the selection criterion. New probabilistic sensitivity 
factors are needed that properly describe the “tail” sensitivities for the response 
function at the final design point. The method of interval analysis (Section 3.4.4) 
is seen as an important tool in error bounding by using acceptable intervals to 
describe the input in a non-probabilistic manner. 

2.3.2 Statement of Work 

An error-bounding tool set is ultimately required. However, the focus of this 
effort is first to define good error bounds by analysis for each of the common 
probabilistic design analysis algorithms in current use for aerospace systems. The 
error bounds are to provide traceable contributions starting from the physical 
problem and proceeding through computational simulation, response surface or 
surrogate model approximations, and then to the final design probability level for 
any defined critical event. 

An industry panel of probabilistic designers is to oversee the development and 
application of the defined bounds to assure that all relevant probabilistic methods 
and problem solving strategies are adequately addressed. 

Recommended error estimate defaults are to be provided for any required input to 
the error bound algorithm. Such defaults are to be defined using available 
statistics or probability theory in a conservative manner. An intelligent interface 
is to be developed that will both guide and assess the user. Such an interface 
should have the ability to impose conservative strategies based on an assessment 
of the user knowledge and expertise. Interval mathematics are to be explored as a 
complementary, non-probabilistic supplement to the probabilistic error 
estimation effort. 
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2.3.3 Software Requirements 

The error bound algorithms are to be documented such that any commercial or 
proprietary probabilistic design system can properly incorporate the algorithms. 
The intelligent PEB tool is to be developed consistent with the following 
requirements. 

All software is to be prepared using “C++” or “JAVA” based on the most up-to- 
date object oriented programming methods. The tool set is defined to mean the 
collection of all software objects required to provide tailored user interfaces, 
analysis modules, data interfaces, module linkages, and output tailoring including 
comprehensive graphics. The tool set is to be provided with non-proprietary 
“data wrappers” such that the input and output results are able to interact with 
text, spreadsheet, free-format, and other forms of data records as well as be able 
to prepare data for commercial analysis codes in the disciplines of structural, heat 
transfer, CFD, and algebraic interpreters. The underlying analysis software is to 
be capable of execution on NT workstations and Unix workstations. The 
interfaces are to be written in Java to be web-enabled and device independent. 

2.4 Response Surface Generator 

2.4.1 Background 

Response surface methods form the basis of all current NDM if one accepts the 
fact that FORM and SORM work with fitted polynomials based on the local 
slope and curvatures of the physical model (which may itself be an 
approximation of the physics). RS’s may be formed from the Taylor series, from 
a design of experiments (DOE) strategy, or from experimental data. Traditional 
RS methods derive from the field of statistics and include not only a physical 
model but information on the statistics of the variables. In the current context we 
do not include the statistics of the RS but admit that experimental RS’s may need 
to include these measures as part of the “assurance level” associated with the RS. 

RS methods may include ties to multidisciplinary design optimization (MDO) 
where approximations to the objective function may be needed, as in linear 
programming and in second order methods. The linkage between NDM and 
MDO is a critical one for future designs of large complex aerospace systems. 

The issues of efficiency and accuracy of RS’s is critical to NDM of design as 
well as to MDO where efficiency is most critical. Efficiency in RS 
approximation may be most important near the mean- value design point and in 
preliminary or conceptual design. Accuracy will become the critical issue in 
NDM near the final probabilistic design point. 

2.4.2 Statement of Work 

A response surface generator tool set is needed. The tool set should provide an 
open interface to information used to characterize physical phenomena. The tools 
should include intelligent systems such as agents, knowledge-based rules, or 
other AI technologies to assist in guiding an efficient computational strategy for 
both probabilistic design and for MDO. The tools must also provide for user- 
controlled, variable-fidelity combinations of speed and accuracy in tailoring the 
RS. The tools must be capable of interaction with the FEB toolset. 

The RS tool set is to include a full complement of DOE facilities for multifactor, 
multilevel “design” variable condition selections. An option to include Taguchi- 
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like DOE strategies focusing on controlled and uncontrolled variables should also 
be included. All interaction terms are to be included for both Taylor series and 
DOE strategies. 

2.4.3 Software Requirements 

All software is to be prepared using “C++” or “JAVA” based on the most up-to- 
date object oriented programming methods. The tool set is defined to mean the 
collection of all software objects required to provide tailored user interfaces, 
analysis modules, data interfaces, module linkages, and output tailoring including 
comprehensive graphics. The tool set is to be provided with non-proprietary 
“data wrappers” such that the input and output results are able to interact with 
text, spreadsheet, free-format, and other forms of data records as well as be able 
to prepare data for commercial analysis codes in the disciplines of structural, heat 
transfer, CFD, and algebraic interpreters. The underlying analysis software is to 
be capable of execution on NT workstations and Unix workstations. The 
interfaces are to be written in Java to be web-enabled and device independent. 

2.5 Probabilistic Model Helper 

2.5.1 Background 

A major limitation in the deployment of probabilistic methods for system design 
is the current burden that the analyst be a skilled “probabilistic engineer.” Such 
individuals at this time are probably numbered in the low dozens and are 
scattered between software firms, universities, government labs, and industry. 

The field is not one that has received much attention in academic curricula 
although the topic is now covered in the most recent undergraduate mechanical 
design texts. Further, the process of preparing and interpreting a probabilistic 
design problem involves the inclusion and assessment of many more sources of 
information and decisions regarding the analysis strategy. 

Considerable work has been done over the past decade to apply probabilistic 
design methods to component design. A major effort in the field of “six sigma" 
design has developed a great deal of data on process and property variations for 
complex mechanical and production systems. Effective linkages to this 
experience would assure that such knowledge and experience can be accessed by 
the current NDM system design analyst. 

Non-traditional methods provide a real basis for the development and 
deployment of intelligent systems to work as probabilistic “robots” or assistants. 
The approach will likely require significant use of adaptive networks, software 
robots, genetic algorithms, expert systems and other non-traditional methods. 

2.5.2 Statement of Work 

An adaptive and intelligent PMH toolset is to be developed that will support high 
quality, high fidelity, and robust non-deterministic design for large, complex 
aerospace systems. The toolset will monitor, assess, guide, and learn. It will also 
have the capacity to operate in a tutorial mode based on an assessment of the user 
skill level. The toolset is to interface to all elements of non-deterministic design 
of such aerospace systems and include elements appropriate to all design stages 
from pre-conceptual to deployed systems. The toolset is to operate on at least 
three levels of user interactions based on a system of expertise evaluation and 
certification. 
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2.5.3 Software Requirements 

All software is to be prepared using “C++” or “JAVA” based on the most up-to- 
date object oriented programming methods. The tool set is defined to mean the 
collection of all software objects required to provide tailored user interfaces, 
analysis modules, data interfaces, module linkages, and output tailoring including 
comprehensive graphics. The tool set is to be provided with non-proprietary 
“data wrappers” such that the input and output results are able to interact with 
text, spreadsheet, free-format, and other forms of data records as well as be able 
to prepare data for commercial analysis codes in the disciplines of structural, heat 
transfer, CFD, and algebraic interpreters. The underlying analysis software is to 
be capable of execution on NT workstations and Unix workstations. The 
interfaces are to be written in Java to be web-enabled and device independent. 

2.6 System Reliability Interface 

2.6.1 Background 

The aerospace system design environment of the future requires the ability to 
interface multiple models of multiple subsystems in an efficient and accurate 
manner. Each failure mode of the system has its own probabilistic model in terms 
of a response surface or surrogate model representation of the physical problem 
together with the associated non-deterministic design variable descriptions. 

Current NASA technology focuses on traditional block-diagram forms for 
representing system reliability. Each block is typically represented by point 
estimates of reliability that are not linked to the underlying physics or to the 
distribution and confidence in the underlying variables. Typical of this important, 
but dated, modeling is the NASA QRAS system developed for the current space 
access system. 

Physics based system modeling provides an information-based opportunity for 
linking the key design parameters and physics-based models together from the 
sub-system/component level to the full system level. Intelligent systems can be 
developed for linking the information and propagating it to the various “top 
level” events in order to provide a powerful environment for calculating and 
managing system level reliability. 

2.6.2 Statement of Work 

The System Reliability Interface toolset will define a working environment for 
managing, linking, and synthesizing system level reliability while providing 
defined “assurance intervals” for the system based on basic design variable 
inputs. The system level operation should be capable of linking to the updating 
tools so that hardware experience, test data, and new information can be used to 
provide updated system reliability and assurance interval results. The System 
Reliability Interface toolset shall have adaptive, reliability networks as the basis 
for recording and calculating system reliability and the marginal distributions for 
all interactive failure modes at every system level. Furthermore, the toolset shall 
have an intelligent means for propagating component level failures to the system 
level while identifying all of the important interactions involved. 

2.6.3 Software Requirements 

All software is to be prepared using “C++” or “JAVA” based on the most up-to- 
date object oriented programming methods. The tool set is defined to mean the 
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collection of all software objects required to provide tailored user interfaces, 
analysis modules, data interfaces, module linkages, and output tailoring including 
comprehensive graphics. The tool set is to be provided with non-proprietary 
“data wrappers” such that the input and output results are able to interact with 
text, spreadsheet, free-format, and other forms of data records as well as be able 
to prepare data for commercial analysis codes in the disciplines of structural, heat 
transfer, CFD, and algebraic interpreters. The underlying analysis software is to 
be capable of execution on NT workstations and Unix workstations. The 
interfaces are to be written in Java to be web-enabled and device independent. 

2.7 System Health Management 

2.7.1 Background 

System Health Management is a philosophy that merges component and system 
level Health Monitoring concepts, consisting of anomaly detection, diagnostic 
and prognostic technologies, with consideration to the design and maintenance 
arenas. Traditionally, Health Monitoring design has not been an integral aspect of 
the design process. This may be partly due to the fact a cost/benefit model of a 
HM system configuration cannot be fully realized at this stage. Without a doubt. 
Health Monitoring technology must “buy” its way into an application. Hence, the 
need exists to extend the utility of advanced no-deterministic system modeling to 
serve as the basis for a virtual environment in which Health Monitoring 
architectures and design tradeoffs can be evaluated and optimized from a 
cost/benefit standpoint. This capability should be present both during the design 
stage and throughout the life of the system. A new HM design strategy should 
allow inclusion of sensors and diagnostic/prognostic technologies to be generated 
from traditional Failure Mode Effects And Criticality Analysis (FMECA) 
information producing an enhanced realization of component design reliability 
requirements at a very early stage. 

Life Cycle Costs can be reduced through implementation of health monitoring 
technologies, optimal maintenance practices and continuous design 
improvement. To date, these areas have not been successfully linked with non- 
deterministic design methods to achieve cost/benefit optimization at the early 
design stage. 

2.7.2 Statement of Work 

An information environment and the necessary tools will be developed in order 
to construct a comprehensive non-deterministic-based System Health 
Management design environment. Design feedback shall be provided by the 
Probabilistic Updater tools. Linkages shall be provided between the design 
information, experimental test data as well as on-line health monitoring data and 
appropriate system models in order to provide continuous design reliability 
improvements, maintenance management, and life cycle cost reductions. The 
System Health Management tools and information environment shall also link to 
traditional system reliability models as derived from FMECA, fault tree, event 
tree and other appropriate system level representations. A key capability shall be 
the ability to operate the system by taking advantage of health monitoring to 
increase overall system performance while maintaining the required system 
reliability. 
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2.7.3 Software Requirements 

The overall software environment must be designed with specific attention paid 
to the diversity of data sources needed for System Health Management, from test 
cell data to fleet operations data to field maintenance data. General purpose tools 
and communication protocols appropriate to System Health Management will be 
defined and developed All software is to be prepared using “C++” or “JAVA” 
based on the most up-to-date object oriented programming methods The tool set 
is defined to mean the collection of all software objects required to provide 
tailored user interfaces, analysis modules, data interfaces, module linkages, and 
output tailoring including comprehensive graphics. The tool set is to be provided 
with non-proprietary “data wrappers” such that the input and output results are 
able to interact with text, spreadsheet, free-format, and other forms of data 
records as well as be able to prepare data for commercial analysis codes in the 
disciplines of structural, heat transfer, CFD, and algebraic interpreters The 
underlying analysis software is to be capable of execution on NT workstations 
and Unix workstations The interfaces are to be written in Java to be web-enabled 
and device independent 

3.0 Discussion 

3.1 Review of Study Process 

The four elements of the study process are shown in Figure 2. The contracted 
Consultant sought to integrate relevant technologies and requirements by making 
personal contacts with each of the indicated entities. The study began by a review 
of efforts relevant to design environments and to non-deterministic, non- 
traditional design at the NASA Glenn Research Center. The Consultant was 
briefed on both technology and safety related programs and initiatives including 
plans for the NPSS environment. The Consultant was given information on the 
NASA Intelligent Synthesis Environment, which he supplemented, by further 
study of the literature and reviews with individuals in that program 
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Figure 2: NDNTM Study Process Elements 

The Consultant is also very actively involved in the deployment of non- 
deterministic design methods and requirements for both low cycle fatigue (LCF) 
and high cycle fatigue (HCF) design of gas turbine engine structures within the 
Air Force via the Engine Structural Integrity Program (ENSIP) handbook (Mil- 
Hdbk-1 783) in support of the Joint Strike Fighter (JSF) and future Air Force and 
Navy Aeropropulsion requirements. The Consultant’s involvement with the Air 
Force is as an advisor on probabilistic design and derives from his extensive 
experience with reliability based design of gas turbines, risk assessment support 
of large fleets of commercial gas turbine engines, and as the program manager 
for the NASA-funded Probabilistic Structural Analysis Methods (PSAM) 
program in the 1983 - 1990 time period. This total experience base has been 
critical in assessing the elements of a future non-deterministic, non-traditional 
design environment. It also opened the doors to talking with key organizations 
and individuals in both the aerospace industry and in technology oriented small 
businesses. 

A literature review of recent publications in various sub-topics within the field of 
non-deterministic, non-traditional analysis methods has been completed. That 
review is given in subject areas that were researched included the following 
topics. 

• Fuzzy logic 

• Possibilistic analysis 

• Interval arithmetic 

• Neural networks 

• Adaptive and Bayesian networks 

• Artificial intelligence (focused applications) 

• Genetic algorithms 

• Probabilistic methods (only new work) 

• Reliability based design (PREDICT) 

• Integrated design methods for large system problems 

• Chaos and complexity theory 

• Health monitoring 

The highlights of findings from the literature review will be given in the next 
section. There were few surprises in that no significant research breakthroughs in 
non-deterministic methods beyond those already deployed appear to be worth 
pursuing. However, it is abundantly clear that there are many non-traditional 
methods that can be brought into the advanced design environment envisioned by 
the ISE initiative. 

The Consultant made a number of visits and teleconferences with key aerospace 
industries and with critical elements of the small business community that are 
active in NDNTM development and deployment. In each visit, the Consultant 
presented a talk on the ISE and discussed how NDNTM are needed to support 
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future design environments. A set of the presentation material from a typical 
presentation is given in the imbedded PowerPoint presentation in Appendix 4.3. 
Selection of that page in any electronic form of this report will invoke the Slide 
presentation. 

The discussions held with industry produced a range of input from verbal ideas 
or priorities for research topics to detailed discussions of new developments 
related to new design environments that share some of the goals of the ISE but 
are more coherently defined and being developed. Some highlights follow. 

• The Boeing Company 

The meeting was held with representatives of various elements of the 
Boeing Phantom Works and Rocketdyne Division. The primary focus of 
the discussion was the Boeing Integrated Vehicle Design System 
(BIVDS). The Consultant was given proprietary information on this 
system. The conclusion is that Boeing is making a very serious attempt 
to define an operating environment that is information based and allows 
three levels of design complexity from conceptual to detailed design The 
major outcome of the meeting was a commitment made by Boeing to 
move strongly to bring non-deterministic methods into BIVDS at the 
earliest possible time. An action team was initiated to lead this effort. 

• Rockwell Science Center 

The Rockwell Science Center has several excellent design system 
initiatives as well as many concepts for elements of ISE. The most 
interesting development for NDNTM is their algebraic design 
environment called Design Sheet™. Design sheet is a constraint 
manager system for working in very large design environments where 
designers must balance many requirements off against each other. 
Examples may be seen at http://www.rsc. rockwell.com/designsheet/ . The 
product calculates constraints and sensitivity equations using the 
MACSYMA™ algebraic manipulation kernel together with some critical 
RSC proprietary algorithms. 

One can envision the operation of Design Sheet™ as a linked set of 
algebraic design rules or response surfaces, linked by common design 
variables. They have developed an 8,000-degree of freedom model 
representing the 3 rd Generation RLV for NASA. The RSC scientist also 
pointed to research at the University of Ohio related to large, 
discontinuous design spaces as an important design environment research 
project. That site is http:/ / www.cis.ohio-state.edu/~chandra/aaai-98 pdf. 

A proprietary White Paper was submitted and is included in this report 
package. 

• GE Aircraft Engines and Corporate R&D 

These corporate entities, along with Engineous Software Inc., Stanford, 
Ohio Aerospace Institute, and others have initiated a major design 
environment initiative with funding participation from NIST. The project 
title is “A Federated Intelligent Product Environment.” There is much of 
great value in this effort ranging from effective means for the 


NAS A/CR— 200 1 -2 1 0976 


19 



development and use of CAD information to non-deterministic design 

[4]- 

A common theme or element in these efforts is generalized information 
processing. A number of non-traditional design methods are deployed 
and all will at some time include a wide range of current non- 
deterministic design analysis methods. All individuals agreed that a key 
need for the future is interoperability issues such that tools can be moved 
between applications. All recognized the need for national initiatives on 
this issue and agreed that NASA would be a logical agent, with others, 
for creating such a national effort. And, all agreed that all would benefit. 

GE recommends that the ISE initiative consider participation in this 
effort. 

• Both Honeywell and Pratt & Whitney have large scale, proprietary 
design environment initiatives They did not provide information on 
those efforts. 

Technology tools and environment research is underway at a number of small 
businesses. The contract effort involved the ones indicated in Figure 2 where Los 
Alamos National Lab is included for some of their effort. The sites were selected 
on the basis of their work in non-traditional system optimization (Engineous) and 
in non-deterministic design (all others). The key areas of related non- 
deterministic tool development expertise are highlighted in the following 
discussion. 

• Applied Research Associates, Inc. 

ARA has been very successful in creating a highly capable software 
interface to various non-deterministic analysis algorithms while using a 
commercial finite element code for the physics modeling (ANSYS ). 
ARA has specific capabilities for the Probabilistic Error Bounder as well 
as response surface technologies. 

• Southwest Research Institute 

SwRI is a leader in probabilistic analysis software with the NASA- 
sponsored NESSUS software product. SwRI identified interval analysis 
using the new Sun Microsystems Inc. Fortran compiler. SwRI is also 
highly capable in areas related to design strategy for system reliability. 

• STI Technologies, Inc. 

STI has extensive experience in response surface technologies and in 
random field and random process representations. 

• Impact Technologies, Inc. 

Impact Technologies has done excellent work in the area of probabilistic 
System Health Management through a number of AF and DOE efforts. 
The SHM tools require extensive use of non-traditional methods for data 
fusion and for data interpretation. 

A White Paper was submitted and is part of this report package. 

• Unipass Software, Inc. 
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Unipass has also developed a commercial probabilistic design code. 

They are particularly interested in research areas that are non-traditional 
applications of probabilistic modeling. 

• Wright State University 

Three white papers on Response Surface modeling were submitted and 
are part of this report package. 

• Los Alamos National Lab 

LANL has been very successful in deploying non-deterministic and non- 
traditional methods to real industrial problems involving total life cycle 
product design as well as a major industrial processing system. In 
particular, they have developed with the Delphi Automotive company a 
design environment called PREDICT. This environment is described by 
several of their presentations now posted at 
http://www.frontier.net/~tcruse/predict . 

Several White Papers were submitted and are included in this report. 

In visits to each of these entities, a presentation was given. The focus on each of 
these visits though was in hearing ideas from each on important NDNTM 
research needs and opportunities. The suggested topics derive from these 
discussions both in general terms as well as the specifics of white papers that 
were submitted by some. Each of the submitted white papers is included in this 
report. 

NASA Efforts that Relate to NDNTM 

Following a presentation of this work to NASA/GRC, the author was made 
aware of other NASA programs that have relations to this effort. These include 
the development of data sharing standards via the international STEP standard 
led by NASA/Goddard. A description of this effort has been taken from their web 
site (http://step.nasa.gov') as follows. 

“The information generated about a product during its design, 
manufacture, use, maintenance, and disposal is used for many purposes 
during that life cycle. The use may involve many computer systems, 
including some that may be located in different organizations. In order to 
support such uses, organizations need to be able to represent their 
product information in a common computer-interpretable form that is 
required to remain complete and consistent when exchanged among 
different computer systems. 

STEP (ISO 10303) is an International Standard for the computer- 
interpretable representation and exchange of product data. The objective 
is to provide a mechanism that is capable of describing product data 
throughout the life cycle of a product, independent from any particular 
system. The nature of this description makes it suitable not only for 
neutral file exchange, but also as a basis for implementing and sharing 
product databases and archiving.” 

NASA Glenn Research Center is supporting another compatible effort focusing 

on developing CAD geometry standards. That work can be viewed at 

http:// www, grc. nasa. gov/W W W/icad and is a cooperative effort through another 
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international consortium known as the Object Management Group (OMG). More 
information on OMG is available at the site http://www.omR.orR . The following 
is the mission statement of the OMG: 

“The OMG was formed to create a component-based software 
marketplace by hastening the introduction of standardized object 
software. The organization's charter includes the establishment of 
industry guidelines and detailed object management specifications to 
provide a common framework for application development. 

Conformance to these specifications will make it possible to develop a 
heterogeneous computing environment across all major hardware 
platforms and operating systems. Implementations of OMG 
specifications can be found on many operating systems across the world 
today. OMG's series of specifications detail the necessary standard 
interfaces for Distributed Object Computing. Us widely popular Internet 
protocol HOP (Internet Inter-ORB Protocol) is being used as the 
infrastructure for technology companies like Netscape, Oracle, Sun, IBM 
and hundreds of others. These specifications are used worldwide to 
develop and deploy distributed applications for vertical markets, 
including Manufacturing, Finance, Telecoms, Electronic Commerce, 
Real-time systems and Health Care. 

OMG defines object management as software development that models 
the real world through representation of "objects." These objects are the 
encapsulation of the attributes, relationships and methods of software 
identifiable program components. A key benefit of an object-oriented 
system is its ability to expand in functionality by extending existing 
components and adding new objects to the system. Object management 
results in faster application development, easier maintenance, enormous 
scalability and reusable software.” 

Both the STEP and OMG efforts are enabling software developments 
complementary to the technology development recommendations included in this 
report. Clearly, the integration of such interoperability standards in any future 
software development research initiatives that are a part of the ISE initiative is 
imperative. 

The preliminary findings of this study were also presented to LaRC. Members of 
the Structures Lab and the ISE program office attended the briefing. The briefing 
was generally well received and no conflicts were identified. Subsequent efforts 
to brief this study at NASA/ ARC to the Intelligent Software Initiative and the 
Design for Quality Initiative were unsuccessful. 

The NASA/MSFC Structures Laboratory (Dr. Roger Townsend has a number of 
projects underway in association with Vanderbilt University under the leadership 
of Prof. S. Mahadevan). The details of those projects are given in the Appendix, 
Section 4.5. 

3.2 Relation of NDNTM to ISE 

The operating premise for the current study is that NDNTM are a subset of the future 
design environment, seen by NASA as the Intelligent Synthesis Environment (ISE). 
Thus, it was deemed appropriate that the study should include reviews of ISE as well 
as studies of the various aerospace industry approaches to the design environment of 
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the future. A typical structure for ISE is the simple interaction diagram shown in 
Figure 3 

Figure 3 is framed around a cultural change in the creativity process Such a change 
cannot occur, as is known to the ISE proponents, unless the ability to access, process, 
and interpret information also undergoes radical change Non-determimstic design, in 
and of itself, poses such a radical change in the design environment Such change is 
slow in coming but it is coming NASA should continue to lead in such radical 
changes as it did in non-determimstic design methods for aerospace systems but the 
current study suggests that NASA will have to make some cultural changes as well 

The current study mostly focused on the Life Cycle Integration and Validation 
element with some interaction with rapid design and synthesis The question posed 
during the many interactions with others was “what are the unique requirements for 
the total life cycle design analysis of complex aerospace systems based on reliability- 
based considerations 0 " More generally, what are the non-determimstic research 
needs to allow such a radical change in the design environment for such system 
design problems 0 



Figure 3: Overview of ISE Initiative 

There are many common elements between the Consultant’s view of NDNTM and 
ISE However, the overwhelmingly common characteristic of both is that both are 
“information processors" (IP). Such an IP focus is attributed to both as both the non- 
deterministic design process and the future total system design process will operate 
on disparate types and forms of data, expert judgment, approximations, and so forth 
It is most effective in terms of modern developments in software engineering to see 
these as information rather than data 

One proposed representation of NDNTM is given in Figure 4. This representation is 
drawn as a design process Based on the literature review and other data sources, the 
most liberal view of NDNTM is shown in a logical arrangement 

• Information Input 

In the past, data for probabilistic modeling was in the form of statistic 
data represented by analytical or empirical distribution functions and 
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their parameters. For the future, input will include a much broader range 
of information. Such information that is subjective or ‘‘fuzzy’’ is 
envisioned as a critical element in non-determimstic analysis The 
elicitation of such fuzzy input is a critical element in the process such 
that expert bias and prior assumptions do not overly influence the 
representation of the information 

• Information Fusion 

This critical junction is where one must convert disparate forms of 
information into a common basis for modeling A critical element in the 
technology requirements here is the translation of properly-acquired 
fuzzy input into sensible probabilistic representations. It is also the place 
where new information must come in to update prior information 



Figure 4: Elements of the ND-NT Design Process 
• ND-NT Methods 

This element contains all of the possible ND-MT methods of modeling 
that were considered in the current program. At this point, the collective 
judgment is that the simulation algorithms and probabilistic methods do 
not need an infusion of new technology However, semi-empirical, 
physics based models of systems based on various networking modeling 
methods is likely to be important, especially for large systems Interval 
mathematics may yet buy its way into design However, this can happen 
only if interval mathematics (discussed in Section 3.4.4) is much faster to 


NASA/CR— 2001 -2 1 0976 


24 





execute than traditional probabilistic methods, given the relatively low 
amount of information that is conveyed and that alternative, probabilistic 
methods can be used for getting the same and more information. Fuzzy 
logic and its attendant elements of membership functions and 
possibilistic analysis is NOT seen as playing a role in this stage of the 
overall design system of the future for aerospace systems. 

Response surface modeling or surrogate (simplified physics) models is 
intrinsic to non-deterministic design. All probabilistic methods use such 
models in one way or another. DOE approaches or Taylor series 
approaches are variations on RS methods. The use of DOE to construct 
such surfaces from empirical data is seen as an alternative to network 
models, although the technologies may merge. 

• ND-NT Outcomes 

This is the heart of the design process valuation. A proper ND-NT design 
environment in the other entities in this diagram will yield much more 
useful information for making decisions for design stages from 
conceptual design to detailed design. 

• Model updating 

A critical element in any design process is updating. Viable design 
operations have evolved means for capturing past experience for new 
designs. Viable risk management programs allow for updating the 
assumed statistics used in the previous modeling. Certainly verification 
and validation of physics models is a real part of good design practice 
today. 

However, the future design environment needs rigorous and sensible 
formalisms for capturing new data as a product is developed and 
deployed such that the data is fed into the process not only for new 
designs but also for managing the design and maintenance of current 
products. 

3.3 Interoperability and Software Environment 

This Consultant is not an expert in software engineering. Nonetheless, the various 
discussions as well as a professional awareness of new trends compel one to include 
a brief set of comments on this topic. Past software development efforts from NASA 
Glenn Research Center have generally, but not always, had a single, “shrink- 
wrapped” code as their objective. It has been stated by NASA/GRC management that 
part of the valuation of leadership from a NASA center is its ability to point to 
industry use of NASA products. This is a very understandable metric. 

However, the design environment of the future is object-oriented software, not 
modular codes. There are big differences. Object oriented code provides many 
features but one of them is adaptivity in run-time. Another is the ability to create 
tools that can be used - again in run time - in various parts of the problem process 
without requiring recompilation of source code. 

If one looks at the recent software engineering initiatives such as the DARPA Robust 
Design Computational System (RDCS) effort at Rocketdyne 

http://www.frontier.net/~tcruse/rdcs/rdcs.htm/ 
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and the Engineous Software called iSight, 
httn://www. engineous com/isi ghtv5. html#products 

one sees a likely paradigm for the future. The computer screen is the design desktop. 
Tools are dragged from one part of the analysis process to another. Datasets are 
dragged and dropped onto an analysis process the designer has just set up on the 
desktop. Such flexibility in the design analysis process is critical to the productivity 
goals set by all major firms. 

3.4 Non-Deterministic and Non-traditional Methods: Status and Opportunities 

The methods selected for review were taken from the AIAA Structures, Dynamics, 
and Materials Conference held in Atlanta in April 2000. A summary of the panelist 
input is given in Appendix 4.2. The list of methods is as follows. Following each 
item is a brief discussion and conclusions for each topic based on the completed 
literature review and various meetings. The titles and order are those that defined the 
AIAA Forum on non-deterministic approaches. 

3.4.1 Neural N etwo rks 

References that were reviewed include [39, 41, 44, 45, 46, and 47], 

Neural nets (NN) have the ability to represent physical responses somewhat 
like a Response Surface used in design. A NN model training process is used 
to establish nodal weights and biases. In general, NN weights do not conform 
to specific physical variables. A NN would be used to model a RS only if one 
had datasets and not a physical model. However, the statistical method for 
constructing RS’s is more robust as far as sorting out noise from model 
behavior and for giving statistics that can be used in constructing assurance 
intervals for error analysis when applying the RS. As pointed out in [39], 
NN’s rapidly lose their attractiveness for large numbers of NN nodes. 

Reference [41] summarizes various uses of networks for data-mining. Such 
applications may be valuable when constructing data models for non- 
deterministic design based on extensive test data. An interesting example of 
the use of NN’s to model a combination of experimental data and varying 
types of modeling results for airfoil design is given in [44]. These authors use 
the NN as a effective way to solve the inverse problem for finding a RS from 
disparate data sources. The also report that their strategy for training results 
in nodal weights that can be tied to specific design variables. 

A Bayesian network is one in which the nodal weights represent conditional 
event probabilities, as discussed in [45]. Such modeling appears to be useful 
for large system reliability problems and provides a direct means for 
simulating the top event and knowing its probabilistic design variable causes. 
They state that the Bayesian network method is preferable to Monte Carlo for 
system simulation for the additional information on conditional probabilities 
that is gained. 

Conclusion: The review indicates that an important role for NN’s is likely to 
be found in the proposed Model Helper and Data Expert. The application of 
NN’s with a strategy to tie the nodal weights to physical variables appear to 
be an effective data mining strategy to support probabilistic input. Bayesian 
networks and other probabilistic learning networks appear to be useful for 
system reliability modeling. Finally, NN’s have a valid and important role in 
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the data gathering and interpretation role in System Health Management. It 
does not seem likely that standard NN’s will be used in the probabilistic 
algorithms. 

3.4.2 Fuzzy Theory 

Fuzzy theory is a “hot” area and is reviewed in References [14, 15, 16, 17, 

18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, and 32], The last of 
these, [32] compares probabilistic and fuzzy modeling for reliability. Fuzzy 
set theory and the closely related field of possibilities are strongly touted as 
the new field for probabilistic modeling. This review does not support that 
position. 

Fuzzy systems are based on the mathematics of fuzzy set theory. The above 
references provide an adequate introduction to these fields. Generally 
speaking, fuzzy systems are used where so-called “crisp” probabilistic 
models do not exist, such as in linguistic representations. Characteristic 
mathematical models are assigned to fuzzy functions that are then combined 
using fuzzy logic. All input variables to the mathematical-physical model are 
given fuzzy representations and the physical model is written using the 
“algebra” of “fuzzy logic” to yield “fuzzy output.” As in the paper by Rao 
[5], one finds that fuzzy physics is violated physics. That is, one loses the 
constraints of the physical model under fuzzy logic formulations. The only 
advice offered by that author is that one must impose one’s physical intuition 
to properly understand the results. This is not scalable as a problem solving 
strategy for large problems. 

Fuzzy logic has been used successfully in a number of industrial control 
system designs that are, in fact, in commercial consumer products today. 
What is the difference? In fuzzy controllers [14, 15, 20, 21 and 25], for 
example, it is clear that fuzzy logic designed control systems for nonlinear 
control problems is highly effective. The difference is that these systems take 
crisp input, operate on the input using fuzzy-logic constructed if-then rules, 
and output crisp results. The advantage of fuzzy controllers appears to be the 
power and flexibility of building the knowledge base of if-then rules and in 
the superior “smoothness” one gets from such a controller. 

Reference [22] applies fuzzy logic to classical reliability engineering. The 
authors contend that this is the right approach to problems where the input is 
estimated from engineering experience and judgment. They also contend that 
this is the right way to represent degraded states of operation, given that the 
word “degraded” is subjective. However, work at the Los Alamos National 
Lab [1], has shown that one can take fuzzy input such as these authors refer 
to and convert that input into probabilistic distributions so that one can 
perform robust probabilistic calculations on these fuzzy-derived 
distributions. The analysts at LANL concur with the Consultant that one 
must use robust non-deterministic algorithms rather than fuzzy algorithms if 
one is to have any confidence and utility from the results. The fact is that the 
whole motivation of the fuzzy logic reliability design community to address 
fuzzy input and output can be addressed in the manner so effectively 
deployed by LANL in their joint PREDICT system work with Delphi 
Automotive. 
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In all the literature reviewed for this program only one remark by an author 
gives on a reason to ponder a bit more about fuzzy logic [16]: The author 
cites sources that argue both sides of this issue and concludes that “there is 
some truth to both sides ...” However, he argues effectively that FL “is a tool 
of enrichment and not replacement ...” Bezdek and Pal [6] are quoted as 
saying that “fuzzy models belong wherever they can provide collateral or 
competitively better information about a physical process.” Bezdek and Pal 
give an example case wherein two bottles of water are lying in the desert and 
are found there by a very thirsty wanderer. The first bottle has a membership 
in the set of potable waters of 0.91 while the other has a probability of being 
potable that is 0.91. In the first case the water shares a high degree of 
characteristics in common with potable water while the second has a 9% 
chance of being totally non-potable (poison!). Which do you drink? 

Conclusion: Fuzzy input is recognized as real based on the Consultant’s 
experience managing risk-based fleet airworthiness programs for commercial 
aircraft engines and based on the LANL/Delphi joint reliability-based design 
program. However, probabilistic algorithms offer the only useful processing 
element for de-fuzzified input data to support complex aerospace system 
design. Fuzzy logic might be deployed in program management decision and 
risk prediction tools. 

3.4.3 Chaos Theory 

A limited review of this subject focused on potential applications to non- 
deterministic design [57], None were found. Chaos theory has a very narrow 
physical interpretation and while chaotic output appears random, it is not. 

The reference indicated is to a thesis that complexity can lead to chaos. 

Conclusion: Chaos theory does not offer critical capabilities to include in 
any recommended NDNTM effort. 

3.4.4 Interval Arithmetic 

References that were reviewed are [35, 36, 37, 38], 

This topic appears to be much more interesting to non-deterministic design. 
Interval arithmetic operates under some very precise algebraic rules and does 
not lead to systems that violate physics. In fact, interval arithmetic was 
developed as a way of representing floating point arithmetic errors. As cited 
in [35], interval arithmetic is now released as a new variable type in the Sun 
Microsystems Fortran compiler. 

Elishakoff and his co-workers have applied interval methods to some 
structural analysis problems. Ref. [7] combines interval analysis with 
stochastic finite element methods (second-moment method) to structural 
frames. Elishakoff applied strict interval analysis methods to computing the 
range of structural natural frequencies in [8]. 

In a related technology to interval methods, Elishakoff and his co-workers 
have utilized what they refer to as convex modeling [9] to compute one-sided 
bounds on structural behaviors such as natural frequencies and buckling 
loads [10], The method is based on defining bounds on uncertain parameters 
as convex sets bounded by ellipsoids. For such cases, convex methods may 
be used to compute a corresponding convex set of results, or upper bounds. 
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Conclusion: Interval methods provide a formal mathematical treatment for 
problems where (1) no input information can be defined beyond upper and 
lower bounds for design variables and/or where (2) response ranges are 
desired for preliminary design purposes. Such methods may be worth 
considering as part of the required error bounds work proposed herein 

Interval arithmetic may offer, especially as a data type, the opportunity to do 
rapid analysis of the output range for design problems. It appears to be 
worthwhile to take some research codes and to convert them to the new type 
of variable to assess how effective this tool would be in defining design 
output range for interval inputs. The approach appears to be one that could be 
applied to error analysis in parts of the non-deterministic design analysis 
process. 

3.4.5 Response Surfaces 

There appears to be some confusion here over terminology as applied to 
probabilistic design. Probabilistic designers have used Taylor series 
expansions of the physical response of systems in terms of the design 
variables to construct locally linear (first-order) and locally quadratic 
(second-order) polynomial fits to model results. Such representations are 
updated as the “design point” moves through the design space (so-called 
advanced mean value - AMV - algorithm). This approach might best be 
called the local-RS approach. 

A second approach, and one currently favored by GEAE and PWA, derives 
from classical statistics [11, 12], These RS methods have been developed as 
part of experimental statistics wherein the experiment is designed with 
varying patterns of high-low values for each identified parameter that can be 
changed. Mathematical methods are used then to define the RS that is most 
likely that defined by the experiment and the statistics of variance regarding 
that fit. One also determines the independence or coupling of the variables in 
the modeled physical response. The use of certain experimental designs leads 
to first order surfaces (with interactions) or second and higher order surfaces. 

While the use of RS’s at GEAE and PWA is a fitting to deterministic and not 
random system responses, both companies define their approach as a RS 
approach consistent with statistical derivations. The reason they use this 
terminology is that they use a variety of experimental designs (DOE) to 
select the points in the design space that are used to evaluate the response 
and from which the first or second order surface (with all interaction terms) 
is fitted. Such a RS method might best be called the DOE-RS method. 

The key in all cases is that the RS is a representation of what is probably a 
model of the physical system. The specific fitting algorithm and probability 
algorithm define the errors in the non-deterministic results. The two RS 
approaches have different characteristics in terms of accuracy (local and 
global) and efficiency. In spite of what GEAE and PWA have taken as 
positions on RS technology theirs is not the last word. 

Conclusion: There are critical technical questions to be resolved regarding 
RS strategies. Most of the questions should be addressed in appropriate 
technical meetings and symposia. There are some technical issues that are 
appropriate for the NDNTM effort. 
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3.4.6 Non-Deterministic Optimization 

Articles reviewed for this subject include [31, 43, 47, 48, 49, 50, 51, 52, 53, 
and 54]. 

All optimization algorithms can and should be non-deterministically applied. 
The two references to the new GEAE design environment [53, 54] clearly 
indicate this combination is the future for GE. Non-probabilistic elements in 
optimization can be in the objective function or in the constraints. The issues 
related to the use of non-traditional optimization strategies (non-gradient for 
example) such as genetic algorithms and simulated annealing have nothing to 
do with non-deterministic issues. They have a lot to do with the size, 
complexity, and nature of the design environment [50, 51]. 

3.4.7 Taguchi 

References [13, 24] are cited. 

There are two elements to Taguchi methods. One is a set of experimental 
designs (patterns of high-lows) as applied in standard DOE methods. DOE 
strategies seek to define a RS by selecting a sub-set of a full factorial design 
procedure in order to reduce the experimental cost. The actual patterns are 
particular to Taguchi and to other developers. The second element is the 
separation of variables into controlled variables and noise or uncontrolled 
variables. When trying to target manufacturing processes using Taguchi 
strategies one seeks to minimize process variability caused by noise factors. 

Conclusion: The use of Taguchi or other, newer strategies is limited for the 
current study to defining RS’s while separating controlled and uncontrolled 
variables. There is no significant research need for work here under the 
NDNTM banner. 

3.4.8 Design of Experiments 

DOE applicability to NDNTM and RS strategies are discussed in the bullets 
above. 

3.4.9 Dedicated Expert Systems 

Conclusion: Expert systems are at the heart of an increasing number of 
applications including the fuzzy logic control systems, as previously 
discussed. Dedicated expert systems will be important parts of NDNTM 
deployed to support future design environments and system health 
monitoring [59, 60], 

3.4.10 Possibilistic 

See fuzzy logic and interval arithmetic. 

3.4. 1 1 Pro bab ilistics 

Conclusion: Any risk critical design application requires probabilistic 
algorithms for robust and reliable results. The consensus of those interviewed 
as well as the opinion of the Consultant is that there are no pressing research 
issues other than error analysis related to the issue of probabilistic 
algorithms. 
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3.4.12 Complexity Theory 

Reviewed articles or web sites are given in [57, 58], 

From the writer’s perspective, advanced aerospace systems may not involve 
complexity in the terms used in [57, 58] even though system modeling is 
highly complicated . Complexity theory arose in thinking about biological 
processes where evolution is involved. Genetic algorithms (GA) for 
optimization appear to use evolution to arrive at an answer - but is that 
related to complexity? This writer contends that it is not related because 
biological evolution leads to time-changes of species indicating that today’s 
answer is not tomorrow’s answer. Properly run GA’s will produce the same 
global optimum for large, complicated problems. 

However, if we view the design process over time from requirements to 
system operation, we often speak in terms of the “evolving design.” We 
make binary choices of configurations, there is no uniqueness but there is a 
strong dependence on the path taken. GA’s cannot model this process as a 
whole because they do not have the full set of options generally available to 
them. A system design problem for which we cannot find a “feasible 
solution” may not have one - or the process may be lacking in robustness to 
the extent that “chaos” is occurring. 

Is there a link between complexity/chaos and probabilistic modeling? This 
writer contends the answer is “no.” We may find it easier to configure 
feasible designs if we allow variability for all of the design variables in the 
sense that the design envelope is fuzzified. We may then find ways to target 
the random design variables to arrive at real, feasible designs. However, 
complexity theory appears to say that it is the shear number of options that 
we are dealing with that is the issue, not the crispness of the parameters used 
for each of the modular options that might be considered. 

Conclusion: The area of complexity theory as applied to the overall design 
process may yield some interesting work. However, it does not seem an 
appropriate area for NDNTM effort. 


4.0 Appendices 

4.1 NDNT Panel Meeting - see CD/ROM files 

The following are comments based on a review of the material presented but not 
on the presentations themselves. Comments will be made on the highlights of the 
presentation based on the other material reviewed and reported herein. Only 
some of the presentations are discussed. The selection is based on those 
presentations that went beyond the use of today’s probabilistic methods or made 
key points related to the purpose of this study. 

• Prof. Ahmad Noor 
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A major focus of his review is on the need for non-deterministic methods in the 
design cycle The basic rationale is the lack of specificity early in the design 


Bounding Uncertainties in Simulation Models 


^SyntnrMs of S; munition Modi 1 ! 


p>{ Tot t tcKi Parameter idi ntific ahon [Updatinn ^Validation 

■ [ /pe r tmt rstol tjatd 
,m(3 its ijfu rrtriinty 

• Mata rrn urt ion 

iHLurp'jr citing _ 

un< ort h intjns 



process The PREDICT program [1 ] was developed with such a contingency and 
offers a rational approach to this problem area. An illustration is given in his 
report for performing interval analysis for a composite shell structure Again, this 
problem is properly handled using today’s probabilistic methods with simplified 
inputs based on the range information The only possible alternative method to 
consider is interval mathematics The elements of selection, parameter 
identification, updating, and validation are fully addressed in today's technology 
based on probabilistic design Again, reference is given to a real design process 
employed in industry [1] 

Professor Noor confirmed in private discussions that the use of fuzzy methods for 
computational algorithms is not recommended due to their inherent limitations 
Probabilistic and interval methods are recommended instead He also makes the 
strong point that existing deterministic analysis methods should serve as the core 
of future NDNTM for the detailed modeling That view is shared by industry and 
this consultant 

• Professor Robert Mullin 

Professor Mullin presents an excellent introduction to interval mathematics for 
the finite element method He also addresses the need for probabilistic 
mesomechanics. The presentation is valuable 

• Dr. David Robinson 
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The conclusion charts from his presentation are valuable to include here. 


oTechnical Issues (2-5 years) 

-new SNL unique algorithms being developed and tested 

-SNL Corrosion Initiative - system level Impact ot microscale evolution of 
corrosion by-products 

waging analyses 

-national power grid vulnerability 

-system level reliability vs component reliability 
Educational issues 

/Technical Issues (Beyond 5 years) 

increased use of NDA methods by engineers 

improved communication between software components 

time dependent behavior 

multiple failure modes 

Educational Issues 

Continued lack of support from educational institutions for qualified 
graduates 


The interesting elements contained here that agree with those recommended in 
this report are improved communication between software elements, multiple 
failure modes (system analysis), probabilistic mesochanics (as seen in his whole 
talk) for complex damage modes, and educational issues. The use of probabilistic 
experts in future software efforts will help here. 

• Professor Efstratios Nikolaidis 

The presentation is an interesting one favoring fuzzy methods. The earlier review 
conclusion on fuzzy methods in this report address his conclusions. 

• Dr. Gene Rogers 

This is an excellent case for effective probabilistic design analysis of space 
vehicles. 

• Dr. Norman Kuchar 

This presentation ties ND methods to GE’s six sigma program 

• Dr. Rob Sues 
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Barriers and Solutions to 
Enable the Potential 

■ Requires specialized expertise -> Need for more 
academic and professional training 

■ Too difficult to implement -> better integration with 
existing CAE tools 

■ Too time consuming to model standardized 
procedures, more demonstration problems 

■ Too time consuming to compute -> numerical 
methods R&D. parallel processing hardware 

■ Immature technology prone to numerical and 
accuracy problems -> numerical methods R&D 
guidelines for application 


The above chart is selected from this presentation The presentation shows 
effective use of probabilities, especially in the area of optimization The above 
barriers and solutions appear to be consistent with many of the themes in this 
report 

4.2 Non-Deterministic Approaches Forum 

Chairmen: Dr. Chris Chamis and Dr. Surin Singhal 
AJAA SDM Conference, April 2000 

Panelists Paul Roth, Bob Kurth, Rob Sues, Eric Fox, Mircea Grigortu 

• Neural Networks: 

(EF) Used with thousands of data points with relatively few outputs Used where 
there is no “structural” model of the system. Can be very difficult to interpret the 
parameters 

(RS) Used anywhere you can use a response surface The method is one of 
generalized curve fitting Can be used to replace a complex model (many degrees 
of freedom) with a simple model (few degrees of output freedom ) 

(BK) Battelle has made effective use of neural networks in the area of avionics 
displays in heads-up systems 

(PR) Cautioned on the improper use of neural networks for extrapolation instead 
of interpolation. 

• Fuzzy Theory 
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(RS) Fuzzy theory is for those not comfortable with probabilistic theory. Same 
goals as probabilistic methods - to bring uncertainty into the design - but not as 
robust. 

(BK) Method is trying to quantify the users lack of knowledge. Battelle does not 
use fuzzy theory in any applications. 

(PR) Have not seen any application of fuzzy theory that could not have been 
converted to probabilistic theory. 

(EF) Fuzzy theory is probabilistic methods using triangular input (for 
distributions). 

(Audience: SS Rao) Fuzzy theory is effective where the input is linguistic and 
not mathematical. 

(Audience: Ahmad Noor) Used where the input data is ambiguous - again a 
linguistic form of input implied. Believes that there are relationships between 
probabilistic methods and fuzzy theory but the latter is not a subset of the former. 

(Audience: Boeing) Fuzzy theory used to emulate human thinking. Employed in 
building self-teaching expert systems. 

(Audience: Dan Ghiocel) Cannot use fuzzy theory when the inputs are correlated. 
Cannot quantify risk. Can be used where qualitative output is desired or 
acceptable. 

(Question raised) Is there a rational transition from fuzzy theory to 
probabilistics? 

• Chaos Theory: 

(BK) Battelle has found applications in mixing theory, navigation problems, and 
- in a probabilistic form - stability problems. 

(PR) Believes that chaos theory will have a significant impact in the future Two 
areas of GEAC application are air-fuel mixing in combustion and in engine 
control. 

(EF) Stock market projections and like problems where there is a lot of noise on 
top of what is otherwise a moderately well controlled process. 

(Audience. Ghiocel) Chaos theory is a deterministic method where non-periodic 
and non-closed output occurs with periodic input. Chaos theory cannot handle 
non-deterministic input. Only the output looks random. 

(Audience Mullen) Applicable to highly nonlinear problems. The method is far 
away from design applicability. 

(Leader: Chamis) Chaos theory is probabilistic simulation where the outcomes 
have equal probability. 

• Interval Arithmetic: 

(PR) No one at GEAC is working in the area of interval arithmetic. The method 
is used to map input bounds into output bounds. Could be used to help the 
probabilistic designer in this way. MC simulation is more effective here than the 
formalisms of interval arithmetic. 
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(EF) If you can use Monte Carlo simulation, you can get all you need from 
interval arithmetic from probability theory. 

(RS) ARA is using the method in developing user manuals to give users expected 
bounds on output. Have made use of it for various simpler problems. For 
complex problems, he recommends using design of experiments with 2 to 5 
factors per interval over the arithmetic. 

(BK) Battelle was very excited about interval arithmetic when first looking at 
developing a fuzzy finite element analysis system. They stopped development of 
the system when they could not overcome the zero-divide problem. They were 
looking for a quantified fuzzy logic system. Some potential for the method 

(MG) Method is not useful for design problems for which failure is caused by 
extreme events, for example, design for time-dependent loads. 

(Audience. SS Rao) Used effectively in tolerance analysis. Automatic operations 
are fast. Output range is so wide that results can be meaningless. Have to use 
physical insight to fine-tune the method. 

(Audience: Mullen) Have to incorporate physics into the method. He has a 
working interval arithmetic FEA code. Has overcome cost problems using Monte 
Carlo. Can reduce large problems to a single FEA run. 

• Response Surfaces: 

(MG) RSs are very useful when the system response is undestood. Can give 
totally wrong results for dynamic problems involving resonance. 

(EF) Response surfaces are best for modestly well-behaved problems. Can test 
the accuracy of models. 

(RS) Use anywhere you don't have a physical model, i.e., can build an empirical 
model, same as curve fitting of observed outputs to inputs. Extrapolations with 
response surfaces can be a problem. Can also be a good surrogate for complex 
computational models as long as you have a strategy for local accuracy near the 
final design point. 

(BK) Response surfaces are very successful for complex physical problems that 
are well behaved. 

(PR) One must pick and choose problems for response surfaces. Rotor mistuning 
with attendant energy localization is not a good problem. Response surfaces can 
capture material variability but not nonlinear physics. 

(Audience: Millwater) Response surface methods appear not to work with large 
numbers of variables, on the order of 20 to 50, that occur in some problems. 
Above 10 design variables, DOE breaks down as there are too many design 
points. Can’t just throw out design variables using sensitivity measures as they 
may become important through interactions or at the final design point. 

(Response: RS) Can use multiple response surfaces with system reliability for 
some of these larger problems. 

• Non-deterministic Optimization: 

(EF) Believes that it is important to use a DOE-based approach. 
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(RS) Believes we should use this method whenever doing optimal design. 
Conditions for optimal design are always probabilistic. Can get “robust design.” 
Can include cost factors. 

(BK) Safety prediction benefits by changing the control limits on the input 
variables. 

(MG) Optimization with reliability constraints is a very promising area for both 
research and applications. Developments are needed to reduce some 
shortcomings, for example, the convergence of the solution to local minima. 

(Audience: Mahadevan) Stated that reliability estimates themselves are random 
Need to combine non-deterministic optimization with robust design methods to 
minimize output uncertainty. 

• Taguchi: 

(RS) Use DOE for process problems. Very similar to probabilistic optimization 

(BK) Used in optimization problems with DOE. 

(PR) GE no longer talks about Taguchi, which is a subset of DOE. 

(EF) PW does a lot of training in Taguchi/DOE to show how badly the Taguchi 
designs do in comparison to classical DOE. Non-isomorphic graph designs are 
better. The original use of Taguchi methods in dealing with uncontrolled 
variables has been replaced by more recent methods 

(MG) A version of experimental design. 

(Audience: Mahadevan) They have combined Taguchi with non-deterministic 
optimization. The method gives qualitative information on which variables to 
use. 

(Reply: EF) Taguchi array can give bad results. His arrays minimize variable 
interactions. 

• Design of Experiments: 

(BK) This is the most important item on the list. The method is used up front in 
any multivariate problem. Reliability problems benefit greatly. 

(PR) DOE is a clever way to minimize variance in estimators. Have to be careful 
though when it comes to interactions. 

(MG) One needs to have some idea of output. Method is useful for variable 
screening. 

(EF) Fractional factorial is very powerful along with Box-Behnken that is best 
for nonlinear problems. 

(RS) DOE is used for building surrogates for complex models. For lots of 
variables, one should probably use Monte Carlo. ARA uses DOE imbedded in 
their code. 

(Question: Mahadevan) DOE is used for non-repeatable data. The response 
surface (as used in probabilistic design; also known as the surrogate model) 
corresponds to repeatable data. Does this change the DOE approach to use? 

(Reply: EF) Box-Behnken or composite designs are best to use if the response 
surface is nonlinear. Lots of process problems have linearities 
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(Reply. RS) Use fractional factorial also. Switch to simpler DOE arrays near the 
critical region for efficiency. 

• Dedicated Expert Systems: 

Panel did not have anything favorable to say here. Professor Rao discussed 
evidence-based models that translate input into belief functions. 

• Possibilistic: 

(MG) Used the example of a coin toss - it is possible for the coin to land on an 
edge but is there any real probability of that event? Possibilistic methods include 
results with very low probabilities. 

(RS) ARA has looked at the method as a way to handle fuzzy logic problems. 

The mathematics are not robust. With the same input one can handle problem 
with more realistic tools. 

• Probabilistics: 

No panel response recorded here except for: 

(EF) Probabilistics can be used with lots or little data or with expert opinions. 

(RS) Use where we have engineered systems and need to do risk assessment, 
predict product reliability, design for reliability, need sensitivity information, in 
new design situations where there is no experience base, and to reduce testing. 

(MG) Probability theory is the most efficient and reliable tool for propagating 
uncertainty through a system. It can be applied when the system and/or the input 
are uncertain. Recent work on fatigue and fracture mechanics shows 
probabilistic tools beyond random variables and random vectors need to be 
considered. For example, random fields are needed to represent the material 
microstructure. Gaussian and non-Gaussian processes are essential for modeling 
various actions on aircrafts. 

• Complexity Theory: 

The panel had little knowledge of this and invited clarification. Chris indicated 
that it is in the ISE plan coming out of NASA/Langley. 

(Audience: Noor) Complexity theory deals with problem modeling where there is 
a large system with a heavy amount of interactions within the system. 

(Audience: Sandia person) Method developed out of work by Los Alamos 
retirees. They have been working in complexity theory since 1988. Are there 
equations that describe common elements of the system? “Self organization” can 
be a part of it. There is a Center for Nonlinear Systems at Los Alamos. Held a 
workshop on predictability of complex systems. They are involved in the 
problem of “swarms” where a large group of autonomous robots interact with 
each other to achieve a defined goal. 
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4.3 NDNT Viewgraph Presentation - ISE-GERD.ppt 
Summarizing the findings and recommendations 
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4.4 White Papers 

White Papers were received in response to the Consultant’s invitation. The following 

pages are cited in terms of the person(s) submitting the White Papers. 

4.4.1 Rockwell Science Center 

See attached, proprietary file - RSC white paper.doc 
This file has restrictions on its cover page. 

4.4.2 Los Alamos National Lab 

4.4.2. 1 Information Integration - Dr. Jane Booker 

Integrating Information for the Analysis of Simulations 

When dealing with the large scale computer simulation codes, it is desirable to 
gain the most information about the code, its imbedded models, and the affect of 
inputs on outputs without running all possible combinations of values. Statistical 
experimental design principles and sensitivity analysis methods can be utilized 
for this purpose: to learn the most from the fewest number of simulation runs. 

When dealing with the analysis of simulation codes, at least two different kinds 
of uncertainty can arise, uncertainties in inputs resulting in uncertainties in 
outputs and modeling uncertainty. The former we call sensitivity analysis, the 
latter is simply "I don't know what my model should be". There are many useful 
methods (e g., Latin Hypercube Sampling) that provide solutions for judiciously 
sampling the input space to learn its effects on the output space. There are also 
many good experimental design techniques to design the runs for the different 
values that the inputs can take on, so that fewer runs are required. The modeling 
uncertainty issue is much less known or studied and is very case/application 
dependent. It is very difficult to design sampling techniques that search through 
and study an infinite number of potentially correct models. 

While these methods provide ways of minimizing the number of runs, they 
cannot produce results from only one or two runs. Carefully designed choices of 
model parameter values and input values can result in understanding how these 
affect the output in sensitivity analysis. The number of these choices can be far 
le ss than the full number required for complete enumeration of all possible 
combinations. Statistical tools using experimental design and sampling 
principles provide ways for making judicious choices. This number could be 
further reduced by taking advantage of other existing knowledge and information 
about the code, the model, the parameters, and their relationships. The principle 
of information integration states that uncertainty can be reduced and better 
estimates obtained by utilizing and combining all available information. A major 
source of information is the knowledge and experiences of the experts in the 
subject matter reflected in the code. There could be other sources as well such as 
results from similar models or codes. We propose to investigate the use of 
information integration methods in conjunction with sampling and design 
methods to utilize all knowledge for reducing the number of required runs for 
sensitivity analysis. 

Likewise, all available information about the model could be used to judiciously 
design code runs for understanding modeling uncertainty. There are many kinds 
of uncertainty that could contribute to the choice and use of models, and these 
would be investigated to provide focus for studying this difficult problem. 
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Model uncertainty is a wide open research area, and we anticipate development 
and modification of information integration methods to understand modeling 
uncertainty and to develop tools for analyzing its effects. 

Jane Booker, Ph D. 

Statistical Sciences Group, MS F600 
Los Alamos National Laboratory 
Los Alamos, NM 87545 
(505) 667-1479 
imb@lanl.gov 

4.4. 2. 2 Statistical Methods and Experiments - Dr. Michael McKay 

Important Research Areas in 

Statistical Theory and Methods for Computer Experiments 

Many critical scientific predictions come from the integration of experimental 
data with calculations from numerical simulation models. Valid methods for 
quantifying uncertainty in scientific predictions require understanding of how 
uncertainties in data and model predictions combine. Three facets of uncertainty 
quantification are (1) experimental error, (2) model prediction uncertainty, and 
(3) combining information. While experimental error is dealt with well by 
existing statistical methodology, methods for analysis of model prediction 
uncertainty as well as methods for combining model predictions with 
experimental data are still in early stages of development. The following 
paragraphs direct attention to the second point, model prediction uncertainty. 

As a setting for model prediction uncertainty, suppose that the prediction y from 
a model m is determined by a vector of input variables x. The input variables 
might define initial conditions of a system being modeled as well as parameter 
values in the rules determining^ from the initial conditions. The term input 
uncertainty is associated with a lack of knowledge about appropriate precise 
input values from which to calculate^. Therefore, x is treated as a random 
variable with a probability density function which quantifies input uncertainty. 
The prediction distribution is the corresponding probability distribution induced 
on y by way of the model m, and characterizes prediction uncertainty. The 
objective of uncertainty analysis is to investigate the relationship between the 
input variables x and the prediction distribution. One part of the investigation is 
to identify (small) subsets of inputs that are important in the sense that they that 
“drive” prediction uncertainty. 

Two common approaches to measuring input importance are differential 
sensitivity analysis and methods based on (linear) regression and correlation 
coefficients. Generally, these approaches are only valid in the neighborhood of a 
“nominal value” or they require that y be approximately linear in x. Furthermore, 
validity of associated importance measures usually requires that the components 
of x be statistically independent. Alternatively, variance-based analysis methods 
provide effective importance indicators (McKay, 1997). The indicators are used 
to evaluate individual inputs as well as subsets of inputs. However, efficient 
algorithms for identifying dominant input subsets of size 1, 2, and so forth are 
needed. Two important research topics follow. 
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Experimental designs for estimation of variance components. Variance 
decompositions require efficient experimental designs for estimation of the 
components of variance for the methods to be feasible. It is proposed to 
investigate traditional variance component estimation in light of sampling 
methods for computer experiments, such as, orthogonal array sampling (Owen, 
1992) and LHS (McKay, 1979, 1995). 

Smart variable selection procedures. When essentially unlimited computer 
runs are feasible, brute-force sequential selection procedures may be adequate to 
select dominant input subsets. However, the shear number possible subsets of 
inputs for models with more than a few inputs can make complete enumeration 
infeasible. It is proposed that smart variable selection procedures which take 
advantage of particulars of the variance decomposition and experimental design 
be developed. It is proposed to try to parallel optimal procedures for subset 
selection in regression, beginning with Hocking (1967). 

Michael D. McKay 
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4.4.2.3 Tools for Elicitation - Dr. Mary Meyer 

Integrated Tool for Eliciting and Representing Knowledge 

One of the most challenging aspects of PREDICT, and indeed any multi- 
disciplinary enterprise, is representing the knowledge that the technical experts 
employ in their problem solving. The proposed tool will accelerate this process 
by guiding the experts, the practitioners, in self eliciting and representing their 
problem-solving knowledge. The tool is comprised of three integrated parts: 1) a 
set of questions that walk the practitioners through defining their technical 
problem and problem-solving processes; 2) a software application that enables 
them to diagram the structure of their problem-solving processes; and 3) a 
customized software application that allows them to input this structure as the 
framework for a prototype knowledge system. We define knowledge systems as 
web-based electronic repositories that have been customized to their users to 
bring together their data, knowledge, and methods in structured, quantitative 
ways to facilitate their problem solving and/or decision making. 

The proposed integrated tool is needed for several reasons. First, self elicitation 
is more efficient than elicitation by another party because it obtains the 
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knowledge from those who possess it and who are most qualified to update it, the 
practitioners themselves. Yet, the techniques for guiding individuals in 
extracting their own knowledge are largely undeveloped. Second, practitioners 
need a graphical tool for representing the structure of their problem solving 
knowledge that is intuitive, transcends their disciplines, and has the capability of 
being transformed to mathematically richer diagrams (e g., conceptual graphs, 
factor complexes). Third, the rapid pace of R&D demands that practitioners be 
able to design the organization of their own knowledge systems, ideally in a day, 
and to upload their files into it structural units. 

Work on portions of the integrated tool has begun. For instance, we have made 
preliminary progress in designing a customizable, self-elicitation process which 
leverages on the PREDICT elicitation methods (Meyer and Booker, 1991 ; 

Meyer, Booker, Bement, 1 999). This latest work is being done in collaboration 
with Dr. Ray Paton, University of Liverpool, U.K. (Meyer and Paton, 2000). We 
have also progressed on the graphical tool for diagramming. Dr. Paton has 
identified the type of diagram, scratch nets, which will be used for eliciting and 
representing. Dr. Paton’s experience and a body of literature document the ease 
of use of this diagram — all users need to identify are the parts of a whole (nodes) 
and how they relate to each other (labeled arcs). In addition, we have 
successfully pilot tested practitioners’ abilities to input the structure for a 
prototype knowledge base and to upload their files into the created structure. 

To continue this work with the aim of creating a prototype integrated tool, we 
anticipate the next steps as: 

1 . Develop, test, and refine the set of self-elicitation questions on pilot 
applications. 

2. Develop the graphic tool by adapting off-the-shelf software, such as 
Visio, to enable users to diagram the nodes and their arcs in scratch net 
form, edit these diagrams, create a data base of the nodes and arcs, and 
transform the scratch nets to more specialized types of diagrams, such as 
hierarchical trees (e g., factor complexes) and or to intermediate forms 
from which conceptual graphs could be created, 

3. Expand the capability of the knowledge system groupware, such as Lotus 
Notes Domino, for accepting user’s complex representations. 

References: 
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M.A. Meyer, M.A. and R.C. Paton, “Interpreting, Representing and Integrating 
Scientific Knowledge from Interdisciplinary Projects,” (Los Alamos National 
Laboratory report LA-UR-00-1448) Theoria et Historia Scientiarum: 
International Journal of Interdisciplinary Studies. Forthcoming. 

Mary Meyer, Ph.D. Ethnology 

Project Leader for Information Integration 

Statistical Sciences Group 

Los Alamos National Laboratory, MS F600 

Los Alamos, New Mexico 87545 

Tel: 505 667-2331 

Fax: 505 667-4470 


4.4. 2.4 Bayesian Optimal Design - Dr. Jane Booker 

Bayesian Optimal Design of Integrated Physical and Computer Experiments 

Both high cost and significant time investment are limiting factors in any 
experimental endeavor. Traditional physical experiments are extremely 
expensive in today's world of highly complex systems. Computational 
experiments are also expensive, but mostly due to the time required to develop 
the computer models upon which they are based. 

Research can be made more inexpensive if a combined experiment, using both 
computational experiments and physical experiments, can be designed. Of 
necessity is determining the relative worth of each type of experimental result so 
that resources can be allocated so as to maximize the amount of information 
gained in an experiment subject to a minimal cost restriction. 

Bayesian experimental design is concerned with the problem of maximizing the 
amount of information gained, taking into account all the uncertainties in 
unknown quantities. Performing the maximization subject to cost and/or time 
budgetary constraint can be accomplished using evolutionary programming 
techniques such as genetic algorithms. This optimization problem is made 
possible by more powerful hardware and innovative algorithms of today's 
computing environments. 

We have developed Bayesian experimental design calculations for the case where 
only one source of data is present. Further, we have performed a combined 
analysis of several different data sources, accounting for possible differences 
(biases) in the data sources which determine relative worth. We have also 
developed genetic algorithms to assess process design prior to production, thus 
eliminating costly mistakes before production begins. Our proposal is to further 
develop Bayesian experimental design for combined experiments under a cost 
and/or time budgetary constraint. 

Jane Booker, Ph.D. 

Statistical Sciences Group, MS F600 
Los Alamos National Laboratory 
Los Alamos, NM 87545 
(505) 667-1479 
imb@lanl.gov 
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4.4.3 Impact Technologies Inc. - Dr. Michael Roemer 

The file is available as a separate document - Impact white paper doc 

4.4.4 Wright State University - Dr. Ramana V. Grandhi 

The file containing three white papers is available as a separate document - 
nasa wp grandhi.doc 

4.4.5 Literature Review for ND/NT Methods 

The following summarizes the literature review made as a part of the contract effort. 

1. Reference [14]: Third thoughts on fuzzy logic. 

The author is an ardent skeptic of fuzzy logic. The context of the article is control systems. 

He was invited to assess the use of fuzzy logic (FL) and he finds its applications to be limited. 
He observes, “when an FL system claims to show great advantages over a conventional 
system, the advantages arise because the conventional system was badly done, or because it 
was highly nonlinear. The use of fuzzy logic in controllers is to provide smoother operations, 
for example However, Pease argues that the use of proportional, integral, derivative (PID) 
controllers can do the same thing. He also cites an authoritative (un-referenced) source as 
saying fuzzy rules grow at a rate of the rules raised to the system dimension. 

The niche in controllers for FL appears to be moderately to highly nonlinear control 
problems. He cites an example problem of supporting a ping-pong ball on a column of air. 
There were some problems with the system though and he observed that optimizing an FL 
system is non-trivial. There is not much supporting data for his comments, which were 
offered to stimulate discussion. 

2. Reference [ 1 5] is an introductory statement for a special issue of the IEEE Proceedings 
entitled Special Issue on FL with Engineering Applications. 

In their introductory sentence, the authors state that “fuzzy logic is a method for representing 
information in a way that resembles natural human communication, and for manipulating that 
information in a way that resembles how humans reason with this information.” Thus we 
have two FL issues to consider: 

• Natural language communication of information. 

• Natural reasoning mode for manipulating the information. 

The authors claim that design-to-market processes may be speeded by the use of FL. 
Application areas have moved beyond controllers to pattern recognition, forecasting, 
reliability engineering, signal processing, monitoring, and diagnosis. 

3. Reference [16] provides a tutorial type of introduction to fuzzy logic systems (FLS). 

An FLS is “able to simultaneously handle numerical data and linguistic knowledge.” FL is a 
nonlinear mapping of input data into scalar output; FL and fuzzy set theory provide the 
mathematical basis for the nonlinear mapping. The use covered in this paper is for “causal” 
systems such as found in engineering. This use is a narrow subset of the methodology. 

An FLS is “a linear combination of fuzzy basis functions and is a nonlinear universal function 
approximator.” The author states that this latter property is one that FL shares with 
feedforward neural networks. The important attribute of the fuzzy basis function is that it can 
be derived from numerical or linguistic knowledge, cast in the form of “if-then” rules. The 
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author states that FL is, to date, “the only approximation method that is able to incorporate 
both types of knowledge in a unified mathematical manner.” 

The author defines two approaches in FL: The first is the model-based approach and the 
second is the model-free approach . In the former, linguistic statements that are converted to 
rules, which are then quantified using FL, represent subjective information. In the latter case, 
the rules are extracted from numerical data and are then combined with linguistic information 
collected from experts, both using FL. The author considers only the second class of 
problems as they can be applied to feedforward neural nets (FFNN) and he compares the two 
methods. The author is concerned about mapping numbers into numbers with front-end 
fuzzifiers and rear-end de-fuzzifiers. 

The Principle of Incompatibility [17] is given by the “founder” of FL: “As the complexity of 
a system increases, our ability to make precise and yet significant statements about its 
behavior diminishes until a threshold is reached beyond which precision or significance (or 
relevance) become almost mutually exclusive characteristics” - or, “The closer one looks at a 
real world problem, the fuzzier becomes its solution.” 

A FLS system can be described as taking crisp input and producing crisp output. In between 
those two is a fuzzifiers of the input data, if-then rules, and a de-fuzzifier output. The rules 
are derived from experts. The rules are written in terms of linguistic variables. An inference 
engine is used to process the if-then rules and produces fuzzy output. 

The membership function for crisp sets is 0 or 1. For FL, the membership function takes on 
values of from 0 to 1 The membership function in FL “provides a measure of the degree of 
similarity of an element in U [the set of allowable values] to the fuzzy subset.” An example is 
car manufacturing if one asks the question of whether the car is domestic or foreign made. In 
crisp logic a rule can fire only if the conditions for the rule are exactly met; in FZ, the rule 
fires “so long as there is a nonzero degree of similarity between the first premise and the 
antecedent of the rule.” The result of the rule firing “is a consequent that has a non-zero 
degree of similarity to the rule’s consequent.” 

Fuzzv logic vs. probabilistic models: The author cites sources that argue both sides of this 
issue and concludes that “there is some truth to both sides ...” However, he argues effectively 
that FL “is a tool of enrichment and not replacement ...” Bezdek and Pal [18] are quoted as 
saying that “fuzzy models belong wherever they can provide collateral or competitively better 
information about a physical process.” Bezdek and Pal give an example case wherein two 
bottles of water are lying in the desert and are found there by a very thirsty wanderer. The 
first bottle has a membership in the set of potable waters of 0.91 while the other has a 
probability of being potable that is 0.91 . In the first case the water shares a high degree of 
characteristics in common with potable water while the second has a 9% chance of being 
totally non-potable (poison!). Which do you drink? 

Fuzzv logic systems are universal anproximators: The question of how well an FLS 
approximates an unknown function is an important question in feedforward neural networks. 
The FFNN is a universal approximator, “which means that a FFNN can uniformly 
approximate any real continuous nonlinear function to an arbitrary degree of accuracy.” The 
same thing has been proved for FLS if that FLS uses “product inference, product implication, 
Gaussian membership functions and height fuzzification.” Other types of FLSs have been 
shown to be universal approximators but not all FLSs fall into this class. Such proofs cover 
existence but they do not say how the FLS is to be constructed. The same is true for FFNNs. 
The existence theorem for FFNN does not tell how many layers of neurons should be used or 
how many neurons should be used in each layer or how interconnected the neurons should be. 
In the case of FLSs the author indicates that one should “design the FLS using representative 
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data that is collected for a specific application.” In fact, one “trains” an FLS in much the 
same way as a neural net is trained by the data. The author does conclude that ‘whatever . . . 
different training algorithm that is used for a neural network can also be used by the FLS. 
Success of any neural network training algorithm depends on the initial values chosen for the 
weight. These weights have no physical meaning for the neural network; hence, they usually 
must be chosen randomly. The parameters of a FLS are associated with membership 
functions for physically meaningful quantities; hence, it is possible o obtain very good initial 
values for them.” The author concludes further “the fuzzification subsystem within the FLS 
lets us handle uncertainty in a very natural way, totally within the framework of the FLSs. To 
date there does not seem to be a comparable way to handle uncertainty in a FFNN.” 

4. Reference [19] describes the unifying framework of adaptive networks. These are said to 
have certain advantages over neural networks. 

Emphasis is given to the use of the “back-propagation” learning rule for artificial neural 
networks. This “universal learning paradigm for any smooth parameterized model” can now 
not only take crisp data but can take linguistic information. It can also adapt itself using 
numerical data to achieve better performance. Neural networks are not able to take linguistic 
information directly. 

Adaptive networks are a “superset of all kinds of neural network paradigms with supervised 
learning capability.” The adaptive network consists of nodes and links arranged in layers In a 
feedforward network each layer links to the next layer whereas in a recurrent network, there 
can be some feedback links. There are no links between nodes in the same layer in a layered 
adaptive network. There can also be a topological network in which nodes only feed those 
nodes with higher numbers in the defined sequence. 

“Usually an adaptive network’s performance is measured as the discrepancy between the 
desired output and the network’s output under the same input conditions This discrepancy is 
called the error measure and it can assume different forms for different applications. 
Generally peaking, a learning rule is derived by applying a specific optimization technique to 
a given error measure.” The backward learning algorithm is based on computing the error 
using a gradient vector whose components involve the parameter at each node. The basic 
concept is to pass a form of the derivative information from the output layer to the input 
layer. 

Special cases of adaptive networks are discussed that have been widely used in the neural 
network literature: the back-propagating neural network and the radial basis function neural 
network. The former is widely used in such areas as pattern recognition signal processing, 
and automatic control while the latter are used to model physiological-like systems where all 
nodes communicate with all nodes. The adaptive network-based fuzzy inference system 
(ANFIS) reported in this paper is similar to the radial basis function network. ANFIS is used 
for nonlinear function modeling, time series prediction, on line parameter identification for 
control systems, and fuzzy controller design. 

Other learning systems can be used. These include gradient free methods such as genetic 
algorithms, simulated annealing, downhill Simplex, and random. Applications have favored 
genetic algorithms. The technology development focus is on “structure determination” 
wherein the network structure, the fuzzy rule sets, etc. can be defined. Speeding up the 
learning algorithm is still needed. 

5. Reference [20]: Fuzzy logic control. 

This is not reviewed due to its narrow focus. The conclusions indicate that the benefits of 
FLC over standard control systems are focused on those control system “policies which 
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combine maximization or constraint enforcement with regulation. Many of the control 
policies implemented are dynamic.” 

6. Reference [2 1 ]: Hardware solutions for fuzzy control. 

The point of this article is that one can implement fuzzy control logic in processors that range 
from supportive to dedicated. “As a general taxonomy, we can identify four classes among 
the different implementation alternatives: 

• Software and hardware solutions with general purpose components. 

• General-purpose processors with instructions for specialized computations. 

• Dedicated fuzzy coprocessors. 

• Fuzzy ASICs capable of stand-alone operations. 

The first alternative is the most widely used and software tools to aid in the development of 
fuzzy controllers have been developed. Special purpose coprocessors have been developed 
and “usually adopt triangular membership functions, flexible rule format (number of 
antecedents and consequents), max-min inference method, 8-b precision and centroid 
defuzzification method. Application specific ASICs have also been developed. 

7. Reference [22]: Application of fuzzy logic to reliability engineering. 

The paper makes the usual statement that system design involves uncertainties and that these 
can be captured using linguistic input. The authors “apply the main concepts of fuzzy logic, 
fuzzy arithmetic and linguistic variables to the analysis of system structures, fault trees, event 
trees, the reliability of degradable systems, and the assessment of system criticality based on 
the severity of a failure and it probability of occurrence.” 

One of the features of the design process that the authors seek to address is “the item whose 
probability of failure is needed often does not exist and it must be estimated based on 
“engineering judgment” or “experience” from “similar” items. The use of fuzzy analysis 
methods including imprecision and approximations, fuzzy set theory, possibility theory, and 
their combination referred to as fuzzy logic, the authors seek to “help restore integrity to 
reliability analyses by allowing uncertainty and not forcing precision where it is not 
possible.” 

The methodology used for system reliability in parallel and series systems is the usual 
membership function approach with assigned mappings of linguistic characterizations such as 
remote, likely, etc. to numerical probability ranges. The output is numerical (translated to 
linguistic) with an assigned “degree of possibility.” The resulting output is like getting a 
probability and a confidence interval in a fuzzy sense. The output needs to be defuzzified. 

An alternative method for computing system reliability for parallel and series systems is that 
of Misra and Onisawa [23], There method is consistent with the max-min operations of 
possibility theory but their results do not provide the correct value when the inputs become 
crisp. 

The authors address the issue of degraded state in their discussion of fuzzy reliability. They 
state that fuzz y sets provides a natural way to represent systems with degradation in that there 
may be many states between fully working and fully not- working. This is achieved by 
defining the memberships in both the working state and the failed state sets. In general, one 
set is the complement of the other. They further state that one can interpret the membership 
functions in “possibilistic” terms as “given that the system is working, the possibility that i 
components are working is” defined mathematically. They draw a weak tie to the Taguchi 
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Loss Function [24] in that the membership function for linear degradation is quadratic. The 
Loss Function evaluates costs associated with component deviations from their nominal 
values as a quadratic function. 

The authors state, “Fuzzy logic provides a more flexible and meaningful way of assessing 
risk. The analysis uses linguistic variables to describe the severity and frequency of 
occurrence of the failure. These parameters are ‘fuzzified’ to determine their degree of 
membership in each input class using membership functions” they provide. “The resulting 
‘fuzzy inputs’ are evaluated using a linguistic rule base and fuzzy logic operations to yield a 
classification of the ‘riskiness’ of the failure and an associated degree of membership in each 
risk class. This ‘fuzzy conclusion’ is then ‘defuzzified’ to give a single risk priority for the 
failure.” 

8. Reference [25]: Industrial applications at GE. 

Fuzzy logic control (FLC) is the principal area of applications. The authors state that FLC 
“has drastically reduced the development time and deployment cost for the synthesis of 
nonlinear controllers for dynamics systems.” FLC has been deployed at GE to turboshaft 
aircraft engine control, steam turbine startup, steam turbine cycling optimization, resonant 
converter power supply control, and data-induced modeling of the nonlinear relationship 
between process variables in a rolling mill stand.” The keys for GE is that they have been 
able to drastically reduce “the development time and deployment cost for the synthesis of 
nonlinear controllers for dynamic systems.” 

The basis for FLC application is the difficulty in synthesizing nonlinear controllers. FLCs are 
“knowledge-based controllers usually derived from a knowledge acquisition process or 
automatically synthesized from self-organizing control architectures [26], The FLC 
represents the nonlinear control surface with a knowledge base that is executed by an 
interpreter or is compiled. GE has reduced the design cycle time during the development 
phase “by using an interactive computing environment based on a high level language with it 
local semantics, interpreter, and compiler.” 

“A hierarchical control scheme permits the decomposition of complex problems into a series 
of smaller and simpler ones. As these simpler problems are solved, typically be using low 
level controllers, the can be recombined to address the larger problem This recombination is 
governed by a fuzzy logic supervisory controller that performs soft switching between 
different modes of operation.” Software maintenance is simplified by being able to switch 
rule bases for different applications, such as changing from a combat environment to a 
training environment. 

The paper goes into some detail on the range of their deployed applications. One of these is a 
neuro-fuzzy system for a critical part of a steel rolling process that cannot be analytically 
modeled. The problem is worked in the forward direction by acquiring operational data, each 
one of which is very expensive. The neuro part of the system is deployed to learn what rules 
govern the data, a reverse engineering problem. The fuzzy part of the system is based on 
generalizing the rules so that table-lookup is not used. This results in a much simpler system. 
The rules can be refined with additional knowledge extracted from the system. 

The major research focus at GE is “aimed at extending adaptation techniques developed in 
other fields to provide automatic tuning of FLCs.” The adaptation techniques they review 
include reinforcement learning, supervised learning, steepest descent, genetic algorithms, and 
reverse engineering/rule clustering. They believe that including other emerging technologies 
such as neural network and genetic algorithms will improve the cost/benefit ratio of FLCs 
when applied to a wide range of complex control problems. 
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9. Reference [27]: Joint Delphi/LANL project on reliability in product and process design 
assurance. 

The report summarizes a joint effort between Delphi Automotive and the Los Alamos 
National Laboratory to develop a software-based mechanical product design system that 
predicts product reliability from conceptual design to product use in the field. Thus, the 
developed “PREDICT” product is a life cycle reliability prediction system that captures 
quantitative and qualitative information about the product during the full design cycle. The 
PREDICT system also computes an uncertainty range on the product reliability, ties the 
reliability and its uncertainty to each of the sub-assemblies, and guides the process for 
reliability improvement. 

A key element of the PREDICT system is the systematic use of expert opinions while 
stripping out bias to the extent possible. A second key element is the use of Bayesian or other 
suitable updating methods for updating the product reliability and its uncertainty based on 
acquired data and usage experience. While specifically designed for use in Delphi products, 
the PREDICT system defines a generic methodology for product design and development 
based on a rational (i.e., not experiential) reliability growth process. 

The PREDICT system makes use of fuzzy input data. Domain experts are used to make point 
estimates of component, subsystem, and system performance in terms familiar to each Each 
expert is also asked for a brief summary of their rational for selecting their estimates. They 
were also asked to provide ranges on their estimates. Such expert input is then transformed 
into distributional values for the analysis. 

10. Reference [28]: Soft Computing and Fuzzy Logic 

The author states that the elements of soft computing include fuzzy logic, neurocomputing, 
and probabilistic reasoning. “The dominant aim of soft computing is to exploit the tolerance 
for imprecision and uncertainty to achieve tractability, robustness, and low solution cost. 
Probabilistic reasoning subsumes genetic algorithms, belief networks, chaotic systems, and 
parts of learning theory. Neuro-computing concerns learning and curve fitting. The concepts 
within fuzzy logic are the use of linguistic variables and the use of fuzzy if-then rules that use 
linguistic variables. Fuzzy logic is logic with fuzzy set theory. 

A central problem in fuzzy logic is how to infer rules from observations. Recent work has 
made important advances toward partial solutions of the problem by applying neural net 
methods with learning based on dynamic [29] and gradient programming [30], By 
representing a fuzzy system in a defined multilayered structure and some learning network 
with backward iteration, one converges on the weights and thereby induces the rules from 
observations. 

1 1. Reference [31]: Using fuzzy logic to drive optimization algorithm 

If one examines optimization algorithms one finds that the algorithms behave as closed-loop 
control algorithms. Rao seeks to use the same benefits of fuzzy logic to build non-linear 
optimizers, in this case the SLP algorithm. 

12. Reference [32]: Probabilistic vs. fuzzy set methods for designing under uncertainty 

The basic hypothesis is that the design data is not well known and therefore we do not have 
perfect “assurance” in the use of probabilistic methods. The desire is to have a conservative 
estimate of the output reliability. The authors state that “if there is sufficient information to 
build accurate probabilistic models of uncertainties, probabilistic methods are better than 
fuzzy set methods.” They go on to assert that fuzzy set methods are better “if little 
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information is available.” The authors use the terms “fuzzy set methods” and “possibility- 
based methods interchangeably.” 

The authors draw several observations and comparisons of fuzzy set and probabilistic 
theories for large parallel and series systems with crisp failure scenarios. They draw two 
conclusions based on these observations. In the first, “a fuzzy set method is likely to 
underestimate the chance of failure of a system with a large number of independent failure 
modes. On the other hand, it can be too conservative in systems for which the failure region is 
small compared to the range of uncertain variables.” They liken the first to the case of having 
a lot of information on independent failure modes where probabilistic methods are favored. 
The second is likened to the case of having limited information on the distributions of the 
independent failure modes. They then draw the next conclusion that “it is easier to determine 
the most conservative fuzzy set model than it is to determine the most conservative 
probabilistic model that is consistent with given information about a problem.” 

Frankly, I find the logic to this discussion pretty shaky. For example, while I agree that it is 
easier to define the most conservative possibilistic model, by their own statement it is clear 
that this result may not be more conservative than a probabilistic model of the system! They 
seem to be hanging the justification of the fuzzy set model solely on the non-rigorous 
contention of “lacking data.” Further, the approach they use is tied to optimizing a design 
using probabilistic or possibilistic input. They contend that the first optimal design may be 
non-conservative - which it certainly could be if optimized to a required safety level and the 
distributions are not absolutely correct. 

The authors “define” uncertainty in the distributions in a soft way by testing the sensitivity of 
the result to changes in the assumed distribution parameters. They further state, “probabilistic 
design ... cannot estimate the sensitivities of the probability of failure with respect to the 
design variables.” This is patently wrong. The simulations used by the authors in their 
demonstrations makes use of a MC loop outside the reliability loop. This is the same way 
others have modeled assurance intervals [33], 

The results of their simulations show that both methods can give unsafe estimates. On 
average, they get more conservative results for the simulations using designs optimized to 
fuzzy set models. However, in neither case do they get any level of assurance nor do they tie 
the results to the most important variable controlling assurance. 

13. Reference [34]: Convex modeling 

This is a very short note and acts now as a place-holder. The contention is that interval 
arithmetic is a form of convex modeling. 

14. Reference [35]: Interval arithmetic. 

The idea for interval math arose out of error analysis. “Error analyses of large scientific, 
engineering, and commercial algorithms are sufficiently complex and labor intensive that 
they are often not conducted. The result is that machine computing with floating-point 
arithmetic is not tightly linked to mathematics, science, commerce or engineering.” 

The interval paradigm is said to have the following characteristics that are considered to be 
valuable. 

• Fallible data can be represented using intervals 

• Any arithmetic computation can be performed using interval arithmetic to obtain 
guaranteed bounds on the set of all possible results. 

• Interval arithmetic can be used to provide bounds on errors from all sources . 
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• A valuable link is http://cs. utep.edu/interval-comp/main.html . The interval arithmetic 
journal is called Reliable Computing, an International Journal devoted to reliable 
mathematical computations based on finite representations and guaranteed accuracy; 
since 1997, published by Kluwer Academic Publishers. 

• There is also a Journal of Convex Analysis f JCA); intervals are special case of 
convex bodies (compact convex sets); there is a strong interrelation between interval 
analysis and convex analysis. 

• Set- Valued Analysis; interval analyis is an important particular case of set- valued 
analysis. 

• Bulletin of ACM SIGSAM (Special Interest Group on Symbolic and Aleebraic 
Manipulations) 

• A Special Issue of "Fuzzv Sets and Systems": Interfaces Between Fuzzy Sets and 
Interval Analysis 

• Special Issue of "Journal of Symbolic Computations" 

• International Journal of Uncertainty, Fuzziness, and Knowledge-Based Reasoning 
qjUFKSl This journal: 

o regularly publishes a special section with abstracts of recent papers on 
applications of interval methods in knowledge representation, and 

o published a special issue on interval methods in representing and processing 
uncertainty. 

The following quote is taken from the “Special Issue ...” cited above. “Fuzzy set theory when 
restricted to sets of real numbers can be approached via intervals and interval analysis. The 
intimate relationship between interval analysis and fuzzy set theory is especially apparent in 
fuzzy arithmetic, fuzzy optimization and some areas of fuzzy mathematical analysis. A 
unimodal fuzzy membership function can be formed from a continuum of intervals 
corresponding to alpha-levels or a continuum of disconnected intervals for multi-modal fuzzy 
membership functions. Intervals are one type of fuzzy set possessing a rectangular 
membership function. This being the case, the issues and mathematical analysis associated 
with fuzzy set theory and interval analysis are mutually relevant.” 

It should be noted that the latest version of the Sun Microsystems Fortran compiler includes 
interval variables. 

15. Reference [36]: Interval arithmetic. 

The article gives a short introduction to interval analysis and its possible applications. It also 
gives an overview of existing programming languages for interval arithmetic. The authors 
state, “the XSC (extended scientific computation) library provides powerful tools necessary 
for achieving high accuracy and reliability. It provides a large number of predefined 
numerical data types and operations to deal with uncertain data.” 

16. Reference [37]: Interval arithmetic (IA) for floating-point computations. 

“Interval analysis concerns the discovery of interval algorithms to produce bounds on the 
accuracy of numerical results that are guaranteed to be both valid and sharp.” The author 
states that IA has been used to develop entirely new algorithms to solve fundamental 
problems that appear to have no other solution. Principal among these are: solving nonlinear 
systems of equations; and nonlinear global optimization (both constrained and 
unconstrained).” [38] 
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While interval analysis has a reduced computational speed, the author says that the 
algorithms are very susceptible to parallel computing. 

17. Reference [39]: Artificial neural networks 

The paper is concerned with the solution of ill-posed problems using neural nets to model 
experimental systems. Size is a big issue as well as noise in the experimental data. “The 
increased interest in these models is motivated mainly by their enormous potential for the 
parallel data processing of multidimensional problems...” The approach taken herein for the 
network is a feedforward multilayer neural network with nonlinear processing elements in the 
network hidden and output layers. A backpropagation supervised learning algorithm is used 
to determine the network’s connection weights and neuron biases. This is supposed to be 
robust and accurate in dealing with noise. The problem is that the number of learning sets 
“increases sharply with the number of cells in the computation grid.” A method like 
simulated annealing is used to determine the weights. 

18. Reference [40]: Integration of reliability and testing 

The approach here is probabilistic design that accounts for uncertainties in the analytical 
model and in the distributions used for the physical variables. Integration of modeling and 
testing is seen as the way to define a “confidence” interval - or what we are not trying to call 
an “assurance interval.” Unfortunately, here the authors are using statistical confidences 
based on the amount of testing. Physical uncertainty, model uncertainty, and statistic 
parameter uncertainties are included. A Bayesian methodology is used for updating with the 
acquisition of failure test data. Both reliability and assurance interval updates are derived. 

19. Reference [41]: Data mining and neural networks 

This is an overview of some strategies used for data mining in five main types of analyses 
classification, prediction, clustering, association and sequence analyses. Classification, 
prediction, and clustering are closely related while association and sequencing are related. 
Each has different methods for different situations including Bayesian modeling, self- 
organizing maps, decision trees, multivariate linear regression, neural nets, and genetic 
algorithms. 

20. Reference [42]: Accurate tail probability prediction 

The paper presents an efficient method for estimating tails as used in Bayesian networks 
when very large confidence intervals are required as in reliability or risk assessment. A 
comparison is made to a fast probability method, which the authors state is the only 
competing method. 

21. Reference [43]: Genetic algorithm search 

The main point made in this paper is that with the use of a GA search with probabilistic 
transition rules such that local minima are avoided. GA works directly with the objective 
function and not with derivatives of it. The authors do point out the enormous computational 
power required to use GAs, a problem only intensified for large systems. 

22. Reference [44]: Combined use of neural networks and response surfaces for airfoil design 

The paper reports on the aerodynamic design of an airfoil using multiple simple response 
surfaces and neural networks to obtain the “advantages of both.” The authors begin with a 
review of the growing literature reporting uses of neural nets for various aerodynamic design 
and modeling applications. A principal concept behind some of the important work is to 
represent the design space using a neural network. This can be seen as a “training derived” 
response surface in design space. Such response surfaces are seen by the various authors as 
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having great utility in multidisciplinary design optimization (MDO), but it is also obvious 
such surfaces may have utility in reliability-based design. 

The authors state “because most design problems in aerodynamics involve a multitude of 
parameters and datasets that often lack structure, neural nets provide a level of flexibility not 
attainable by other methods.” They also state that design rules can be incorporated along with 
high fidelity CFD solutions. A feed-forward, gradient-based learning algorithm is used along 
with perturbation data about the starting design point to train the neural network. 

In the current paper, the authors build a “parameter-based partitioning of the design space” 
using neural nets to link variables of interest to key design parameters and simple 
polynomials for the remainder of the design parameters. This approach significantly reduces 
the factorial design space problem, according to the authors. There is no mention of a more 
efficient DOE approach. 

The authors also address the problem of a final design that is a large distance from the initial 
design point. They do this by constructing a series of response surfaces in the design space. 
The neural net advantage would appear to be in providing an efficient means for solving the 
inverse problem of a response surface definition (as opposed to a matrix inversion requiring a 
square system) for large data sets. Their methodology would also appear to provide weights 
that are physically based so that the response surface can be linked directly to the design 
variables for the reliability problems. 

23. Reference [45]: Feature saliency in neural networks 

The authors develop a formal means for assessing the importance of design features in trained 
neural nets in terms both of the physical sensitivity and the error sensitivities. It would be 
interesting to contrast this with the DOE and ANOVA methods. 

24. Reference [46]: Bayesian network in a reliability problem 

In the words of the authors “A Bayesian network is a directed acyclic graph in which nodes 
represent random variables and links represent direct probabilistic influences. The variables 
depicted in the BN represent key parameters characterizing the system being modeled.” The 
network links define conditional probability relations between causes and effects based on the 
network topology The total network then represents the system reliability in a direct, 
physical manner. The topology of the BN is constructed using intermediate variables so that 
the conditional probabilities are kept simple, using only two parent nodes for each child node. 

As in standard reliability modeling, the number of independent variables is kept small by 
neglecting those with small likelihoods. The BN provides an easy mechanism to perform 
“what-if’ analyses on the system. Conditional sensitivities are obtained. The what-if analysis 
can involve changing the prior distributions to see the effect of individual element changes on 
the posterior distribution. 

The paper gives an example for a power system and shows that the constructed BN gives the 
same answer as “other methods” which I take to be a simulation based method. However, the 
authors state that the ability to have a direct computation of the posterior distribution is 
unique to the BN method. BNs also provide a direct evaluation of conditional probabilities. 
Commercial software was used to propagate the conditional distributions through to the top 
event. 

25. Reference [47]: Globally convergent optimization with neural nets 

The authors have devised a particular way of constructing NNs such that the constructed net 
is guaranteed to be globally convergent to solutions of problems with bounded or unbounded 
solution sets. The authors review the literature on the use of NNs for gradient and non- 
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gradient optimization algorithms that are widely used. The computational advantage to NNs 
is their inherent parallel structure. The proposed algorithm includes all proposed gradient and 
non-gradient methods as special cases. 

Unfortunately, the authors make the statement that the derived NN is NOT unique for the 
same optimization problem How then have we found a global optimal design? Apparently 
we have not done so. All that is assured is that the algorithm is convergent to A solution, not 
the solution. There does not appear to be a special capability here for use in reliability 
problems. 

26. Reference [48]: Conceptual design using genetic algorithm for trade selection 

The principal hypothesis behind the paper is that preliminary design for complex systems is 
often focused on feasibility rather than optimality, given the problem size and methods used. 
This certainly corresponds to my own design experience on the PW2037 engine where the 
preliminary design did not consider the downstream impact of early decisions on non- 
optimality at the sub-system level. As a result, the high-pressure turbine was very non- 
optimal, but it is flying today. To resolve the problem from the authors’ perspective, they 
address the issue of generating a wide range of alternatives at the conceptual design step. 

The focus in this paper is on combinatorial design where many discrete configurations are to 
be considered. Classical, continuous variable methods do not work for such design 
optimization problems. The general field espoused in this article is that of meta-heuristics and 
includes genetic algorithms, neural nets, simulated annealing, tabu search, and hybrids of 
these. Genetic algorithms work with a population of possible designs rather than evaluating 
an ordered sequence of designs and are therefore highly parallel in structure. 

The key element in this paper is the use of a spreadsheet environment to represent each of the 
subsystems as performance models. The GA then is able to select from the multiple design 
concepts by accessing the spreadsheet. 

My question is whether or not one can be effective for reliability-based design using a GA 
and a system integration model. 

27. Reference [49]: Optimizing to fuzzy, possibilistic constraints 

Instead of usual reliability-based optimization where the reliability constraints are crisp, this 
application is one in which the constraints are treated as possibilistic ranges. Is there an 
application for us? 

28. Reference [50]: Nonstandard methods for optimization 

This paper gives a reasoned discussion of the limits of standard, gradient-based methods vs. 
the non- traditional methods such as simulated annealing, genetic algorithms. Tabu search, 
and rule-based expert systems. For the many variants of random methods, the author says 
“the computational requirements associated with the use of these methods in problems with 
increasing dimensionality continues to be excessive, necessitating their use in conjunction 
with function approximations. The author reviews the combination of algorithmic and rule- 
based system used by iSIGHT (Engineous Software). 

29. Reference [51]: Exploring large design spaces 

A critical element in the ISE picture for the future is the intrinsic element of large scale 
design problems. This paper reports on an innovative and certainly non-traditional method for 
exploring these large design spaces. The goal is not optimization, but exploration of multiple 
configurations (non-continuous design space) with some hope that good designs will not be 
overlooked. A key question to me is whether or not and/or how to bring reliability into this 
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process. Traditional design optimization is performed in the design parameter space. In his 
case all design variants are continuously coupled. Conceptual design may include both 
parameter space design and configuration changes through analogy brainstorming. Such a 
design process is discontinuous. 

Furthermore, the authors state “hill climbing [traditional optimization] techniques, by 
requiring that a single evaluation function be defined, preclude explicit, local reasoning about 
tradeoffs among multiple performance criteria. The essential element of the approach is the 
use of a device library with characteristics for each device, a set of domain critics that assess 
the devices as they are selected, a design-seeker algorithm, and constraints. The library has 
extensive performance modeling, the critics are expert systems, and the seeker is a high level 
filtering process. Much of the paper is concerned with the filtering algorithm but all elements 
are addressed and demonstrated on an example conceptual design study. 

It would appear that a “telescoping” design simulation method is required to assure the 
systems interfaces of the elements taken from the device library as well as the physical 
plausibility of the devices. Reliability attributes need to be developed and linked through this 
process. 

30. Reference [52]: Review of computational intelligence 

The article is the editorial overview of a special edition and touches on fuzzy, neural, and 
evolutionary computation, and on hybrid computational intelligence systems that combine 
some of these. Useful set of references is provided. 

31. Reference [53]: Integrated design systems 

32. Reference [54]: More integrated design systems 

These two closely couple reports concern the effort at GE to develop a highly integrated 
design system for gas turbine engine design. The basis of that system is what they call the 
Intelligent Master Model (IMM) environment and it supports highly concurrent engine 
design. Optimality, robustness, and quality are key objectives for the design environment 
attributes. 

Elements in IMM include the following: 

• CAD systems based on features 

• High- and low-fidelity analysis modules 

• Multidisciplinary design optimization (MIX)) 

• Robust design 

• Reliability prediction 

The IMM is “a fusion of knowledge-based engineering with top-down product control, 
conventional master modeling, and the linked product model environment.” “The IMM can 
contain part dependencies, geometric and non-geometric attributes, manufacturing 
producibility, and cost constraints, as well as access to external databases, and be integrated 
with proprietary codes, through the linked model environment.” 

The Master Model (MM), at the lowest geometric level, consists of basic features used in 
solid modeling - holes, fillets, etc. that can be extruded and combined with the usual Boolean 
operations. The software is highly modular with interoperable tools with data-wrappers 
providing interfaces. Knowledge-based engineering provides the ability to define, develop 
and use design rules at any level. 
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The system supports both the inclusion of quality metrics derived from the GE six-sigma 
effort as well as probabilistic modeling using Monte Carlo, response surfaces from design of 
experiments (DOE), and fast probability integration. Robustness seeks to reduce the 
sensitivity of the design to uncontrolled variables, in a Tuguchi-like sense. 

A highly developed part of the environment is the geometric modeling using “feature-based” 
design tools. The features can then be linked according to rules and design practices captured 
in simple spreadsheet models. The system integrates CFD, heat transfer, and structural 
modeling modules. 

The written material demonstrates a close affinity with the goals set for the ISE. Non- 
traditional and non-deterministic elements are significant parts of the developing product. 

33. Reference [55]: Possibilistic reliability 

The authors have created an algorithm that derives from fuzzy logic but closely mimics 
probabilistic reliability methods. By converting all variables into standard normal form and 
treating possibilistic input with Gaussian participation functions, the authors are able to find a 
most possible point by a simple and rapid algebraic operation. Cumulative possibility plots 
and possibility density plots are derived. 

The authors cite speed as a great advantage for their algorithm. 

34. Reference [56]: Bayesian updates 

The paper reviews the various Bayesian approaches that might be used in structural reliability 
analysis when performing quantitative risk assessments. It is a reasonable review of Bayesian 
methodology. They conclude that “it is often difficult to use the classical Bayesian approach 
in decision-making as the resulting uncertainty intervals are so large.” However, they further 
say that “if the fully Bayesian approach is adopted, the output results are expressing the 
analysis group’s total uncertainty related to observable quantities, and it possible to present a 
clear message.” The fully Bayesian approach “will provide the probabilities of the uncertain 
events that are relevant in the specific situation of decision-making. The probabilities are total 
in the sense that they incorporate all types of uncertainty.” 

35. Reference [57]: War and Chaos 

This article was read to gain insight into the area of complexity. In the words of this author 
complexity theory “allows scientists to make mathematical models of events in which the 
inputs do not necessarily have a direct, or linear, relation to the outcome.” He further equates 
unpredictability with chaos. The author, a physicist, hoped to use complexity theory to 
predict the point of instability - in the social context of chaos leading to war. 

“Complexity may be defined as the set of deterministic theories that do not necessarily lead to 
long-term prediction. Such theories are still mathematical and deterministic. . . . but the 
structure of the mathematics is such that we cannot obtain the future values implied by the 
theory just as a result of a compact, well-defined manipulation of the present values. The 
calculation requires the actual computational stepping through all the intermediate values of 
the system variables between ‘now’ and ‘then.’ Complexity theories thus depend on the 
complete ‘path’ taken by the system between its beginning and end points. As such, they are 
sensitive to all perturbations that may have an impact on the system as it evolves in time.” 

To support this, I further quote from the author. The example he is citing concerns planetary 
motions. The universe might be thought of as having a complicated structure. “The solution 
of the differential equations [of planetary motion] in closed form gives the structural variables 
- position and velocity - at any time, in terms of well-determined trigonometric functions. 
Plug in the time you want, and out come the desired structural variables. There is no need to 
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compute them at intermediate times. Modify the initial conditions slightly and the output 
variables come out slightly differently. Thus the entire future of the system, to any specified 
accuracy, is completely contained in, and obtainable from, the initial parameters and the 
theory via finite explicitly specified numerical procedures. The closed continuous 
mathematical form of the solutions to the theory’s equations means that output is 
continuously related to input - there can be no surprises, no disorder and no complexity.” 

36. Reference [58]: A web site on complexity theory 

The author of this web site defines “Complexity is the property of a real world system that is 
manifest in the inability of any one formalism being adequate to capture all its properties. It 
requires that we find distinctly different ways of interacting with systems. Distinctly different 
in the sense that when we make successful models, the formal systems needed to describe 
each distinct aspect are NOT derivable from each other.” 

37. Reference [59]: Gas turbine health monitoring, diagnosis, and prognostics 

This PWA paper reports on a comprehensive system based on statistical inference, artificial 
neural networks (ANNs), Bayesian updating, and expert systems. The system does not use 
probabilities to set ranges on variables but instead uses exponential averaging of engine data. 
ANNs are built on a sub-system basis to enforce physical behavior to the nodes and training 
processes rather than the usual ANN condition where the nodal weights have no physical 
interpretation. This approach aids in fault root-cause identification. “A Bayesian type [of] 
statistical evidence approach is used to reflect the uncertainties in the rule based system.” The 
knowledge based expert system provides the interpreter for making prognostic inferences 
based on the information derived from the ANN. 

38. Reference [60]: Advanced diagnostics and prognostics for risk assessment 

The elements of the approach are “Statistical-based anomaly detection algorithms, fault 
pattern recognition techniques and advanced probabilistic models for diagnosing structural, 
performance and vibration related faults and degradation.” The prognostic modules use 
probabilistic, physics-based models of system behavior to get the expected range of failure 
lives for risk-based decisions. The probabilistic physics-based models can also be used in 
making probabilistic anomaly decisions. Fuzzy logic is used to perform real-time sensor 
validity analysis. 

4.5 Application Projects for NDM sponsored at Vanderbilt University by the 
NASA/MSFC (cf. Prof. S. Mahadevan) 

The on-going probabilistic methods research at Vanderbilt University is sponsored by NASA 
Marshall Space Flight Center through two GSRP (Graduate Student Research Program) 
fellowships (Joshua Hall and Andrew Stoebner). The two GSRP fellowships fit into the overall 
objective of the MSFC Structural Dynamics Laboratory (technical lead: Dr. John Townsend) to 
familiarize its engineers with probabilistic analysis and design methods and to develop 
demonstration problems of interest to MSFC engineers. Three tasks accomplished under this 
funding are described below. In addition to these three tasks, MSFC organized two three-day 
short courses on probabilistic methods, taught by Prof. Mahadevan. The first course covered basic 
probabilistic techniques — first-order reliability methods, Monte Carlo simulation, response 
surface methods, and system reliability analysis. The second course covered advanced techniques 
in probabilistic finite element analysis, space and time variability problems, advanced Monte 
Carlo methods, and detailed demonstration problems. The three tasks are as follows: 


1 . Probabilistic Response Surface Analysis of Solid Rocket Booster Aft-Skirt 
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NASA engineers at Marshall Space Flight Center (MSFC) have applied probabilistic methods to 
assess the reliability of the aft-skirt of the Solid Rocket Booster (SRB). The objective of the 
response surface analysis in this task is to develop a closed form solution for the stresses in the 
aft-skirt. This solution can then be used as a performance function for further component level 
reliability analyses on the aft-skirt, or as a simplified model of the skirt to be used in a system 
level analysis of the entire Space Shuttle. The response surface model is validated with the 
previous solutions obtained by the engineers at MSFC. 

The aft-skirt structure is the mechanism by which the SRB’s, and the rest of the Space Shuttle 
assembly are attached to the Mobile Launch Platform (MLP). The skirt is constructed of 2219- 
T87 aluminum. Each skirt has four hold-down post forgings, which are butt-welded onto the skin 
panels. The failure of the skirt is known to occur at a point at the bottom of the weld region 
between the forged post and the skin of the skirt. The primary contributor to the failure of the 
skirt is the bending load caused by the Space Shuttle Main Engines (SSME) at start-up. Just prior 
to launch the SSME’s are test fired. This thrust load is eccentric with respect to the aft-skirt, and 
thus induces a forward sway of the Space Shuttle Assembly. The maximum load to the skirt 
occurs at the maximum point of deflection due to the SSME thrust. 

The engineers at MSFC used a NASTRAN finite element model of the aft-skirt assembly to 
calculate the state of stress at the critical weld region. An improved design using brackets was 
also investigated. A large amount of strain gauge data was gathered from the shuttle flights 
following the Challenger accident. This data was used to characterize the statistics of the loading 
variables to be used in the probabilistic analysis. The NESSUS/NASTRAN interface was utilized 
to perform the probabilistic analysis. The results of the prior analyses are compared with those 
obtained using the proposed response surface methodology, and it is shown that the response 
surface method provides excellent savings in computational effort for this problem. 

2. Probabilistic Analysis of Solar Concentrator 

This task presents the consideration of uncertainties and the application of probabilistic 
computational methods to the dynamic analysis of polyimide inflatable cylinders used in solar 
thermal propulsion. The basic concept behind solar thermal propulsion is to utilize solar energy as 
a means of heating a propellant to provide thrust at increased specific impulse. Thrust is produced 
by expanding the heated propellant through a nozzle. No combustion occurs, and the thrust level 
is low. For this reason, solar thermal propulsion systems are mainly applicable for orbital transfer 
vehicles. 

A prototype inflatable solar concentrator consists of a torus/Iens assembly supported by three 
struts. This concentrator is constructed of Kapton polyimide film, with epoxy as the primary 
adhesive for joints. The Fresnel lens of such a concentrator assembly would focus sunlight into a 
collector near the fixed ends of the struts. Solar energy stored in the collector could be utilized to 
heat the propellant. 

The inflatable cylindrical struts are critical components of this assembly. Therefore, their 
dynamic and static behavior has been investigated both experimentally and analytically by 
engineers at NASA Marshall Space Flight Center. The polyimide film material used for 
construction of the struts is highly nonlinear. Its elastic modulus varies as a function of frequency, 
temperature, and the level of excitation. It has been already observed that the elastic modulus 
decreases with increasing frequency, in free-free as well as cantilevered configurations. 

The purpose of this task is to apply probabilistic computational methods to this problem. There 
are several types of uncertainties in these structures, such as material properties, boundary 
conditions, loading conditions, ambient temperature, damping etc. This task develops statistical 


N AS A/CR— 200 1 -210976 


59 



information about these uncertain variables using the test data already compiled at Marshall 
Space Flight Center, and demonstrates the use of this information in the dynamic analysis of the 
inflatable struts. Two types of NASTRAN finite element models are used, with (1) beam 
elements that incorporate the frequency dependent nature of the elastic modulus, and (2) shell 
elements. Two types of probabilistic analyses are performed. The first one is to compute the 
statistics of the natural frequencies and the load response. The second one is to compute the 
reliability of the structure under various performance criteria such as strengh, stability, stiffness 

etc. 

3 Reliabilitv-Based Robust Design 

This task combines two major concepts of reliability-based design and robust design. Reliability- 
based design attempts to maximize the reliability of a system. Robust design tries to minimize 
the performance variation when the operating conditions change. An ideal design will be robust 
and reliable. The method proposed in this study investigates reliability design and robust design 
methods and combines both types of techniques into a single objective function. In this study, 
robustness is defined as variation in the reliability estimate. This technique will allow the 
designer to satisfy reliability and robustness requirements with one function, rather than looking 
at them separately. Also, it is shown how to measure each factor’s influence on the reliability and 
robustness of the system. 

Three types of design are used to demonstrate the proposed method. Parameter design uses the 
mean values of the random variables in the design. Tolerance design uses the standard deviations 
of the variables in the design. Combined design uses both means and standard deviations in the 
design. These three designs are applied to four demonstration problems: truss design, automotive 
leaf spring design, weld design, and throttle design. 

4.6 Original Statement of Work 

Purpose of the Study: To prepare a report to NASA on Non-Deterministic (i.e., non- 
traditional) Methods for Design. The report will include a review of the methods in terms of 
basic methodology, an identification of who is doing the work or making applications of the 
methodology, and how the methodology is being used. The report will also identify 
application opportunities for NDM in advanced aerospace applications and will define future 
development needs. The report is to include an executive summary, review of methodologies, 
applications, future needs and opportunities, bibliography, and an Appendix with site visit 
interview summaries. 

Proposed Statement of Work: The proposed effort will begin by developing a list of sites 
and/or persons that spans a wide range of the leading work and topics. An initial proposed list 
is appended to this proposed statement of work. The PI will then contact each site or person 
to define for them the nature of the NASA-directed study. The PI will obtain written versions 
of the work being pursued in terms of papers and reports for early review of the methodology 
and/or applications. Following this review the PI will call each selected site or person to 
arrange on-site visit with key investigators and/or application engineers. The PI will develop 
interview notes that summarize the work being done by the methodology developers and 
application engineers. 

The written material and the interviews will form the basis for a written final report to NASA. 
The work will begin as soon as authorized. The proposed schedule for the interviews and the 
final report is as follows: 


Project initiation: Review final proposed list of contacts and visits. 
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Phase I: Make contacts and solicit participation in the study; request 

relevant written material [2 weeks] 


Phase II: Review written material and develop regional travel plans; notify 

NASA Program Manager of each planned trip and proposed 
contacts. [Estimated schedule based on 2 weeks to prepare for 
and complete each regional visit: 10 weeks] 

Phase III: Prepare final report draft for NASA approval [2 weeks] 

Initial proposed regional travel: Potential travel sites that are initially proposed are given 
below. The proposed effort is to make five multisite visits to maximize the information return 
per trip. The PI will arrange the schedule for each multisite visit to maximize the number of 
specific/individual company/lab/university sites per trip. The proposed travel also includes up 
to two trips to Cleveland to add to the knowledge base for the report and to present an oral 
review of the report. An overview of the study is to be used as the basis for a keynote lecture 
to be given to the AIAA SDM conference in Atlanta GA during the first week in April. 

Visit to the Northeast via NYC: 

• GE Corporate R&D in Schenectady NY : fuzzy control, uncertainty modeling and 
analysis. 

• Impact-Technology (Dr Mike Roemer) in Rochester NY; fuzzy neural nets for 
system health monitoring and prognostication. 

• STI (Dr. Dan Ghiocel) in Rochester NY; probabilistic fields and response surface 
technology. 

• UTRC in East Hartford CT (Dr. Wally Orisamolu): fuzzy systems, applications of 
reliability-based design to non-aerospace systems. 

• Cornell in Ithaca NY (Prof. Grigoriu). advanced probabilistic methods. 

Visit to central US via Cincinnati: 

• UDRI in Dayton OH (Drs. Pete Hovey and A1 Berens); random processes; 
probabilistics for applied NDE. 

• Wright State University in Dayton (Dr. Ramana Grandhi); fast probability 
methods. 

• AFRL/PR (Ted Fecke); AF applications and development plans for non- 
deterministic design. 

• GEAC, Cincinnati OH (Drs. Paul Roth and Dennis Corbly): probabilistic design 
methods for gas turbine engines. 

• VPISU, Blacksburg VA (Prof. E. Nikolaidis): fuzzy sets; potential istic methods. 

• ANSYS, Pittsburgh PA (Dr. Stefan Reh). non-deterministic design via CAE. 

Visit to Florida: 
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• PWA (Eric Fox and Chuck Annis): probabilistic design methods for gas turbine 
and rocket engines. 

• U. So. Florida (Prof. S. Rao): fuzzy sets, interval math. 

• Florida Atlantic University (Prof. I. Elishakoff): Convex methods. 

Visit to Los Angeles: 

• Unipass Inc. in LA (Dr. M. Khalessi): probabilistic design technology 

• Boeing/Rocketdyne in Canoga Park (Drs. S Mehta, and Rajagopal, and others): 
non-deterministic design methodology for engineering systems; DARPA program 
review 

• AlphaStar Inc. . Probabilistic design methods for advanced composites 

• UC Berkeley (Prof. Der Kiureghian); advanced probabilistic methods in CE 
Visits to Southwest and South-central. 

• Honeywell Systems (Dr. Michael Gorelik): applications of probabilistic methods in 
certified gas turbine engine design; advanced methods. 

• Sandia Natl. Lab in Albuquerque: advanced non-deterministic design methods. 

• Southwest Research Institute (Dr. Justin Wu): advanced probabilistic design 
methods and applications. 

If a particular site visit is not possible, the PI will follow up with extended phone calls to 
answer specific technology questions for the report. 
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